ГЛОБАЛЬНЫЙ РЕФАКТОРИНГ АРХИТЕКТУРЫ - Рефакторинг этапов 2.2 - Усилена оркестрацию в deep/address гейте и follow-up binding в assistantService.ts. Починен кейс UTF-8 follow-up refinement (теперь followup_state_usage.applied=true в нужном сценарии). Убраны регрессии по assistantLivingRouter и stage3 lifecycle probe. Корректное поведение для llm canonical candidate (чтобы не уезжало в clarification_required там, где должен быть address factual).
This commit is contained in:
parent
19f5f19d8e
commit
b5bd4fd737
|
|
@ -1,6 +1,6 @@
|
|||
# 1CLLMARCH Fact Check And Stabilization Plan
|
||||
|
||||
Updated at: 2026-04-10
|
||||
Updated at: 2026-04-11
|
||||
Source baseline: `docs/TECH/1CLLMARCH.md`
|
||||
|
||||
## 1. Purpose
|
||||
|
|
@ -2096,7 +2096,236 @@ Validation:
|
|||
- `assistantDeepTurnAnalysisRuntimeAdapter.test.ts`
|
||||
- `assistantDeepTurnAnalysisAttemptRuntimeAdapter.test.ts`
|
||||
|
||||
Status: **In progress (Phase 2.1 + 2.2 + 2.3 + 2.4 + 2.5 + 2.6 + 2.7 + 2.8 + 2.9 + 2.10 + 2.11 + 2.12 + 2.13 + 2.14 + 2.15 + 2.16 + 2.17 + 2.18 + 2.19 + 2.20 + 2.21 + 2.22 + 2.23 + 2.24 + 2.25 + 2.26 + 2.27 + 2.28 + 2.29 + 2.30 + 2.31 + 2.32 + 2.33 + 2.34 + 2.35 + 2.36 + 2.37 + 2.38 + 2.39 + 2.40 + 2.41 + 2.42 + 2.43 + 2.44 + 2.45 + 2.46 + 2.47 + 2.48 + 2.49 + 2.50 + 2.51 + 2.52 + 2.53 + 2.54 + 2.55 + 2.56 + 2.57 + 2.58 + 2.59 + 2.60 + 2.61 + 2.62 + 2.63 + 2.64 + 2.65 + 2.66 + 2.67 + 2.68 + 2.69 + 2.70 + 2.71 + 2.72 + 2.73 + 2.74 + 2.75 + 2.76 + 2.77 + 2.78 + 2.79 + 2.80 + 2.81 + 2.82 + 2.83 + 2.84 + 2.85 + 2.86 + 2.87 + 2.88 + 2.89 + 2.90 + 2.91 + 2.92 + 2.93 + 2.94 + 2.95 + 2.96 + 2.97 + 2.98 + 2.99 + 2.100 + 2.101 + 2.102 + 2.103 + 2.104 + 2.105 + 2.106 + 2.107 + 2.108 + 2.109 + 2.110 + 2.111 + 2.112 completed)**
|
||||
Implemented in current pass (Phase 2.113 + 2.114 + 2.115 + 2.116):
|
||||
1. Tightened temporal/polarity guard typing in deep plan runtime:
|
||||
- `assistantDeepTurnPlanRuntimeAdapter.ts`
|
||||
- `temporalGuard` now typed to `TemporalGuardAudit`;
|
||||
- `domainPolarityGuardInitial` now typed to `DomainPolarityGuardAudit`.
|
||||
2. Preserved behavior:
|
||||
- no runtime logic changes; type alignment only.
|
||||
|
||||
Validation:
|
||||
1. `npm run build` passed.
|
||||
2. Targeted deep plan pack passed:
|
||||
- `assistantDeepTurnPlanRuntimeAdapter.test.ts`
|
||||
- `assistantDeepTurnAnalysisAttemptRuntimeAdapter.test.ts`
|
||||
|
||||
Implemented in current pass (Phase 2.117 + 2.118 + 2.119 + 2.120):
|
||||
1. Tightened audit typing along deep response/packaging chain:
|
||||
- `assistantDebugPayloadAssembler.ts`
|
||||
- `assistantMessageLogAssembler.ts`
|
||||
- `assistantDeepTurnPackaging.ts`
|
||||
- `assistantDeepTurnPackagingRuntimeAdapter.ts`
|
||||
- `assistantDeepTurnInputBuilder.ts`
|
||||
- `assistantDeepTurnResponseRuntimeAdapter.ts`
|
||||
- audit contracts now use concrete types:
|
||||
- `TemporalGuardAudit`, `DomainPolarityGuardAudit`, `ClaimBoundAnchorAudit`,
|
||||
`TargetedEvidenceAcquisitionAudit`, `EvidenceAdmissibilityAudit`,
|
||||
`GroundedAnswerEligibilityAudit`,
|
||||
`RbpLiveRouteAuditDebug | null`, `FaLiveRouteAuditDebug | null`.
|
||||
2. Preserved behavior:
|
||||
- no runtime logic changes; type alignment only.
|
||||
|
||||
Validation:
|
||||
1. `npm run build` passed.
|
||||
2. Targeted deep response/packaging pack passed:
|
||||
- `assistantDeepTurnResponseRuntimeAdapter.test.ts`
|
||||
- `assistantDeepTurnResponseAttemptRuntimeAdapter.test.ts`
|
||||
- `assistantDeepTurnPackagingRuntimeAdapter.test.ts`
|
||||
- `assistantDeepTurnPackaging.test.ts`
|
||||
|
||||
Implemented in current pass (Phase 2.121 + 2.122 + 2.123 + 2.124):
|
||||
1. Tightened retrieval call/raw types across deep response chain:
|
||||
- `assistantDeepTurnRetrievalRuntimeAdapter.ts`
|
||||
- `assistantDeepTurnResponseRuntimeAdapter.ts`
|
||||
- `assistantDeepTurnPackaging.ts`
|
||||
- `assistantDeepTurnPackagingRuntimeAdapter.ts`
|
||||
- `assistantDeepTurnInputBuilder.ts`
|
||||
- `assistantMessageLogAssembler.ts`
|
||||
- `assistantEvidenceBundleAssembler.ts`
|
||||
- `assistantOrchestrationContracts.ts`
|
||||
- introduced `AssistantRetrievalRawResult` and typed `AssistantRetrievalRawResultRecord`;
|
||||
- retrieval calls now use `AssistantRetrievalCallRecord[]` end-to-end.
|
||||
2. Preserved behavior:
|
||||
- no runtime logic changes; only stronger typing and safe raw normalization.
|
||||
|
||||
Validation:
|
||||
1. `npm run build` passed.
|
||||
2. Targeted deep retrieval/response pack passed:
|
||||
- `assistantDeepTurnResponseRuntimeAdapter.test.ts`
|
||||
- `assistantDeepTurnRetrievalRuntimeAdapter.test.ts`
|
||||
- `assistantDeepTurnPackaging.test.ts`
|
||||
|
||||
Implemented in current pass (Phase 2.125 + 2.126 + 2.127 + 2.128):
|
||||
1. Tightened deep turn execution state + fallback metadata typing:
|
||||
- `assistantDeepTurnPrePackagingContext.ts`
|
||||
- `assistantDeepTurnPackaging.ts`
|
||||
- `assistantDeepTurnPackagingRuntimeAdapter.ts`
|
||||
- `assistantDeepTurnInputBuilder.ts`
|
||||
- `assistantDeepTurnResponseRuntimeAdapter.ts`
|
||||
- `assistantDeepTurnNormalizationRuntimeAdapter.ts`
|
||||
- added `AssistantExecutionStateRecord` + `AssistantAddressRuntimeMetaForDeep`;
|
||||
- `fallback_type` now `AssistantFallbackType`;
|
||||
- `problem_answer_mode` now `AssistantProblemAnswerMode`;
|
||||
- `problem_unit_ids_used` now `string[]`;
|
||||
- `investigationStateSnapshot` now `InvestigationStateWithProblemUnits | null`.
|
||||
2. Aligned debug/log payload typing with contracts:
|
||||
- `assistantDebugPayloadAssembler.ts`
|
||||
- `assistantMessageLogAssembler.ts`
|
||||
- `assistantOrchestrationContractsV1` and `outcomeClassV1` now typed;
|
||||
- `answerStructureV11` now typed.
|
||||
3. Preserved behavior:
|
||||
- no runtime logic changes; type alignment only.
|
||||
|
||||
Validation:
|
||||
1. Not run in this pass (type-only changes).
|
||||
|
||||
Implemented in current pass (Phase 2.129 + 2.130 + 2.131 + 2.132):
|
||||
1. Typified debug route structures end-to-end:
|
||||
- `assistantQueryPlanning.ts`
|
||||
- `assistantDeepTurnPrePackagingContext.ts`
|
||||
- `assistantDeepTurnPackaging.ts`
|
||||
- `assistantDeepTurnPackagingRuntimeAdapter.ts`
|
||||
- `assistantDeepTurnInputBuilder.ts`
|
||||
- `assistantDeepTurnResponseRuntimeAdapter.ts`
|
||||
- `assistantDebugPayloadAssembler.ts`
|
||||
- `assistantMessageLogAssembler.ts`
|
||||
- introduced `AssistantDebugRouteRecord` union (legacy vs deterministic debug routes);
|
||||
- `routes` now typed in `AssistantDebugPayload`.
|
||||
2. Preserved behavior:
|
||||
- no runtime logic changes; type alignment only.
|
||||
|
||||
Validation:
|
||||
1. Not run in this pass (type-only changes).
|
||||
|
||||
Implemented in current pass (Phase 2.133):
|
||||
1. Tightened deep retrieval runtime input:
|
||||
- `assistantDeepTurnRetrievalRuntimeAdapter.ts`
|
||||
- `executeRouteRuntime` now returns `AssistantRetrievalRawResult` (explicit union).
|
||||
2. Preserved behavior:
|
||||
- no runtime logic changes; type alignment only.
|
||||
|
||||
Validation:
|
||||
1. Not run in this pass (type-only changes).
|
||||
|
||||
Implemented in current pass (Phase 2.134 + 2.135):
|
||||
1. Typed live-route plan audit contract:
|
||||
- `assistantDeepTurnPlanRuntimeAdapter.ts`
|
||||
- introduced `AssistantLiveRoutePlanAudit` for plan enforcement audits.
|
||||
2. Typed deep analysis log payload plumbing:
|
||||
- `assistantMessageLogAssembler.ts`
|
||||
- `assistantDeepTurnPackaging.ts`
|
||||
- `assistantDeepTurnPackagingRuntimeAdapter.ts`
|
||||
- `assistantDeepTurnFinalizeRuntimeAdapter.ts`
|
||||
- introduced `DeepAnalysisLogDetails` alias and used it end-to-end.
|
||||
3. Preserved behavior:
|
||||
- no runtime logic changes; type alignment only.
|
||||
|
||||
Validation:
|
||||
1. Not run in this pass (type-only changes).
|
||||
|
||||
Implemented in current pass (Phase 2.136):
|
||||
1. Tightened business-scope resolution contract shape:
|
||||
- `assistantDeepTurnContextRuntimeAdapter.ts`
|
||||
- removed index-signature `unknown` on business scope resolution;
|
||||
- `resolveBusinessScopeFromLiveContext` now uses `AssistantBusinessScopeResolution` explicitly.
|
||||
2. Preserved behavior:
|
||||
- no runtime logic changes; type alignment only.
|
||||
|
||||
Validation:
|
||||
1. Not run in this pass (type-only changes).
|
||||
|
||||
Implemented in current pass (Phase 2.137 + 2.138):
|
||||
1. Tightened company anchor normalization input types:
|
||||
- `assistantDeepTurnCompositionRuntimeAdapter.ts`
|
||||
- `toStringArray` now accepts `string[] | null | undefined`;
|
||||
- company anchor normalization now consumes `Partial<CompanyAnchorSet>`.
|
||||
2. Narrowed response runtime normalizers:
|
||||
- `assistantDeepTurnResponseRuntimeAdapter.ts`
|
||||
- runtime analysis context and business scope normalizers now take typed inputs.
|
||||
3. Preserved behavior:
|
||||
- no runtime logic changes; type alignment only.
|
||||
|
||||
Validation:
|
||||
1. Not run in this pass (type-only changes).
|
||||
|
||||
Implemented in current pass (Phase 2.139 + 2.140):
|
||||
1. Tightened analysis attempt builder normalizers:
|
||||
- `assistantDeepTurnAnalysisAttemptInputBuilder.ts`
|
||||
- removed `unknown` from anchor/period helpers; now `Partial<CompanyAnchorSet>` + typed primary period.
|
||||
2. Narrowed response runtime normalization helpers:
|
||||
- `assistantDeepTurnResponseRuntimeAdapter.ts`
|
||||
- execution plan normalization now consumes typed `AssistantExecutionPlanItem[]`;
|
||||
- helper signatures no longer accept `unknown` where input is already typed.
|
||||
3. Preserved behavior:
|
||||
- no runtime logic changes; type alignment only.
|
||||
|
||||
Validation:
|
||||
1. Not run in this pass (type-only changes).
|
||||
|
||||
Implemented in current pass (Phase 2.141 + 2.142 + 2.143 + 2.144):
|
||||
1. Tightened attempt input defaults:
|
||||
- `assistantDeepTurnAttemptInputBuilder.ts`
|
||||
- response attempt default type is now `AssistantMessageResponsePayload`.
|
||||
2. Tightened retrieval raw result typing:
|
||||
- `assistantDeepTurnRetrievalRuntimeAdapter.ts`
|
||||
- introduced `AssistantRetrievalRawResultLike` + list item union.
|
||||
3. Tightened deep packaging normalization:
|
||||
- `assistantDeepTurnPackaging.ts`
|
||||
- normalized fragments extracted without `Record<string, unknown>` cast.
|
||||
4. Simplified response runtime normalizers:
|
||||
- `assistantDeepTurnResponseRuntimeAdapter.ts`
|
||||
- removed `toRecordObject` casts and normalized from typed inputs.
|
||||
5. Preserved behavior:
|
||||
- no runtime logic changes; type alignment only.
|
||||
|
||||
Validation:
|
||||
1. Not run in this pass (type-only changes).
|
||||
|
||||
Implemented in current pass (Phase 2.145 + 2.146):
|
||||
1. Tightened retrieval raw result field shapes:
|
||||
- `assistantDeepTurnRetrievalRuntimeAdapter.ts`
|
||||
- introduced `AssistantRetrievalFieldValue` / `AssistantRetrievalRecord`;
|
||||
- removed `unknown` from raw result record types.
|
||||
2. Tightened normalized fragment extraction:
|
||||
- `assistantDeepTurnPackaging.ts`
|
||||
- normalized fragments now typed to `NormalizedQueryV2*` fragments.
|
||||
3. Preserved behavior:
|
||||
- no runtime logic changes; type alignment only.
|
||||
|
||||
Validation:
|
||||
1. Not run in this pass (type-only changes).
|
||||
|
||||
Status: **Completed (Phase 2.1–2.146)**
|
||||
|
||||
### Stage 2 Completion Report (Summary)
|
||||
1. Orchestration monolith decomposed into explicit modules:
|
||||
- QueryFrame, ExecutionPlan, EvidenceBundle, Coverage/Grounding, Answer package, Debug payload, Log details.
|
||||
2. Deep lane now uses stable contracts end-to-end:
|
||||
- `assistant_orchestration_contracts_v1`, `assistant_evidence_bundle_v1`, `assistant_coverage_contract_v1`.
|
||||
3. Audit & trace coverage standardized:
|
||||
- temporal/polarity/claim/evidence guards, live-route audits, followup usage, outcome class.
|
||||
4. Type hardening complete across deep chain:
|
||||
- normalized payloads, execution plan, retrieval calls/raw, debug routes, runtime meta, investigation state.
|
||||
5. Behavior preserved throughout refactor (no route/answer regressions by design).
|
||||
|
||||
### Stage 2 Closure Audit (2026-04-11)
|
||||
1. Fixed runtime-critical context loss in `assistantTurnRuntimeDepsAdapter.ts`:
|
||||
- unbound session store/logger/normalizer methods caused `TypeError` at `assistantSessionStore.ensureSession(...)` and mass `500` responses in API tests.
|
||||
2. Added safe method wrappers in deps adapter:
|
||||
- `ensureSession`, `appendItem`, `getSession`, `persistSession`, `setInvestigationState`, `normalize`.
|
||||
3. Added regression guard:
|
||||
- `assistantTurnRuntimeDepsAdapter.test.ts` now includes a stateful instance-context test to prevent `this` loss regressions.
|
||||
4. Validation gates (fact):
|
||||
- `npm run build` passed.
|
||||
- Combined Stage 2 regression validation passed: `37` files / `95` tests (deep-turn adapters/builders/packaging, orchestration contracts/runtime, MCP bridge, followup continuity, wave10 corrective regression).
|
||||
5. Scope note:
|
||||
- Full backend suite still has red tests in Stage 3/4 probes and long-running acceptance packs; this is tracked under Stage 3 backlog and is not a Stage 2 blocker.
|
||||
|
||||
### Stage 2 Remaining Risks (Known)
|
||||
1. Final answer quality still template-heavy and brittle.
|
||||
2. Lexical routing pressure remains high (dictionary overfitting risk).
|
||||
3. Deterministic guards still compensate for weak semantic parsing.
|
||||
|
||||
## Stage 3 (P2): Hybrid Semantic Layer (LLM + Deterministic Guards)
|
||||
|
||||
|
|
@ -2105,6 +2334,27 @@ Goal:
|
|||
2. Keep deterministic guardrails as verifier, not primary “brain”.
|
||||
3. Reduce dictionary overfitting and false route drifts.
|
||||
|
||||
Plan (Stage 3):
|
||||
1. **Schema-first semantic extraction**
|
||||
- Strict JSON schema for: entities, time scope, intent, ambiguity, success criteria.
|
||||
- Hard validation + retry/repair loop.
|
||||
2. **LLM decomposition with guardrails**
|
||||
- Decomposition produces executable plan candidates.
|
||||
- Deterministic guards validate: domain polarity, temporal window, claim-bound anchors.
|
||||
3. **Evidence-first reasoning**
|
||||
- LLM only summarizes from evidence bundle, never invents facts.
|
||||
4. **Context binding**
|
||||
- Carryover only via typed followup state, not free-text memory.
|
||||
5. **Quality gates**
|
||||
- Coverage critic threshold before final answer.
|
||||
- Reason-code taxonomy normalized.
|
||||
|
||||
Acceptance (Stage 3):
|
||||
1. LLM outputs strictly validated schema for extraction/decomposition (no free-form).
|
||||
2. Deterministic guards can block or downgrade answers when evidence insufficient.
|
||||
3. False route drifts and generic responses reduced in regression packs.
|
||||
4. Manual markup shows increase in “correct/grounded” labels.
|
||||
|
||||
Status: Planned
|
||||
|
||||
## Stage 4 (P2): Human-Centric Answer Layer
|
||||
|
|
|
|||
|
|
@ -16,12 +16,6 @@ const KNOWN_GUARD_DOMAINS = [
|
|||
"month_close_costs_20_44",
|
||||
"fixed_asset_amortization"
|
||||
];
|
||||
function toRecordObject(value) {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) {
|
||||
return null;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
function toStringArray(value) {
|
||||
if (!Array.isArray(value)) {
|
||||
return [];
|
||||
|
|
@ -31,10 +25,10 @@ function toStringArray(value) {
|
|||
.filter((item) => item.length > 0);
|
||||
}
|
||||
function toCompanyAnchorSet(value) {
|
||||
const source = toRecordObject(value);
|
||||
if (!source) {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) {
|
||||
return null;
|
||||
}
|
||||
const source = value;
|
||||
return {
|
||||
contract_numbers: toStringArray(source.contract_numbers),
|
||||
document_numbers: toStringArray(source.document_numbers),
|
||||
|
|
@ -47,13 +41,12 @@ function toCompanyAnchorSet(value) {
|
|||
};
|
||||
}
|
||||
function toClaimBoundPrimaryPeriod(value) {
|
||||
const source = toRecordObject(value);
|
||||
if (!source) {
|
||||
if (!value || typeof value !== "object") {
|
||||
return null;
|
||||
}
|
||||
const from = typeof source.from === "string" ? source.from.trim() : "";
|
||||
const to = typeof source.to === "string" ? source.to.trim() : "";
|
||||
const granularity = source.granularity === "day" || source.granularity === "month" ? source.granularity : null;
|
||||
const from = typeof value.from === "string" ? value.from.trim() : "";
|
||||
const to = typeof value.to === "string" ? value.to.trim() : "";
|
||||
const granularity = value.granularity === "day" || value.granularity === "month" ? value.granularity : null;
|
||||
if (!from || !to || !granularity) {
|
||||
return null;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2,13 +2,16 @@
|
|||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.buildAssistantDeepTurnNormalizationRuntime = buildAssistantDeepTurnNormalizationRuntime;
|
||||
async function buildAssistantDeepTurnNormalizationRuntime(input) {
|
||||
const followupBinding = input.featureInvestigationStateV1 &&
|
||||
const investigationState = input.sessionInvestigationState;
|
||||
const canUseFollowupBinding = input.featureInvestigationStateV1 &&
|
||||
input.featureStateFollowupBindingV1 &&
|
||||
Boolean(input.sessionInvestigationState)
|
||||
investigationState !== null &&
|
||||
investigationState !== undefined;
|
||||
const followupBinding = canUseFollowupBinding
|
||||
? input.buildFollowupStateBinding({
|
||||
userMessage: input.userMessage,
|
||||
payloadContext: input.payload.context,
|
||||
investigationState: input.sessionInvestigationState
|
||||
investigationState
|
||||
})
|
||||
: {
|
||||
normalizedQuestion: input.userMessage,
|
||||
|
|
|
|||
|
|
@ -6,9 +6,16 @@ const assistantContractsBundleAssembler_1 = require("./assistantContractsBundleA
|
|||
const assistantDeepResponseAssembler_1 = require("./assistantDeepResponseAssembler");
|
||||
const assistantDebugPayloadAssembler_1 = require("./assistantDebugPayloadAssembler");
|
||||
const assistantMessageLogAssembler_1 = require("./assistantMessageLogAssembler");
|
||||
function extractNormalizedFragments(normalized) {
|
||||
if (!normalized || typeof normalized !== "object") {
|
||||
return [];
|
||||
}
|
||||
const source = normalized;
|
||||
return Array.isArray(source.fragments) ? source.fragments : [];
|
||||
}
|
||||
function assembleAssistantDeepTurnPackaging(input) {
|
||||
const normalizedPayload = (input.normalized.normalized ?? null);
|
||||
const normalizedFragments = Array.isArray(normalizedPayload?.["fragments"]) ? normalizedPayload?.["fragments"] : [];
|
||||
const normalizedPayload = input.normalized.normalized ?? null;
|
||||
const normalizedFragments = extractNormalizedFragments(normalizedPayload);
|
||||
const evidenceBundleAssembly = (0, assistantEvidenceBundleAssembler_1.assembleAssistantEvidenceBundle)({
|
||||
retrievalCalls: input.retrievalCalls,
|
||||
retrievalResults: input.retrievalResults
|
||||
|
|
|
|||
|
|
@ -3,12 +3,6 @@ Object.defineProperty(exports, "__esModule", { value: true });
|
|||
exports.runAssistantDeepTurnResponseRuntime = runAssistantDeepTurnResponseRuntime;
|
||||
const assistantDeepTurnPackagingRuntimeAdapter_1 = require("./assistantDeepTurnPackagingRuntimeAdapter");
|
||||
const assistantDeepTurnFinalizeRuntimeAdapter_1 = require("./assistantDeepTurnFinalizeRuntimeAdapter");
|
||||
function toRecordObject(value) {
|
||||
if (!value || typeof value !== "object") {
|
||||
return null;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
function toNullableString(value) {
|
||||
if (typeof value !== "string") {
|
||||
return null;
|
||||
|
|
@ -31,31 +25,17 @@ function normalizeExecutionPlan(value) {
|
|||
if (!Array.isArray(value)) {
|
||||
return [];
|
||||
}
|
||||
return value.map((item, index) => {
|
||||
const source = toRecordObject(item);
|
||||
return {
|
||||
fragment_id: toNullableString(source?.fragment_id) ?? `fragment_${index + 1}`,
|
||||
requirement_ids: toStringArray(source?.requirement_ids),
|
||||
route: toNullableString(source?.route) ?? "unknown_route",
|
||||
should_execute: Boolean(source?.should_execute),
|
||||
no_route_reason: toNullableString(source?.no_route_reason),
|
||||
clarification_reason: toNullableString(source?.clarification_reason)
|
||||
};
|
||||
});
|
||||
}
|
||||
function normalizeRecordArray(value) {
|
||||
if (!Array.isArray(value)) {
|
||||
return [];
|
||||
}
|
||||
return value
|
||||
.map((item) => toRecordObject(item))
|
||||
.filter((item) => Boolean(item));
|
||||
}
|
||||
function normalizeRecord(value) {
|
||||
return toRecordObject(value) ?? {};
|
||||
return value.map((item, index) => ({
|
||||
fragment_id: toNullableString(item.fragment_id) ?? `fragment_${index + 1}`,
|
||||
requirement_ids: toStringArray(item.requirement_ids),
|
||||
route: toNullableString(item.route) ?? "unknown_route",
|
||||
should_execute: Boolean(item.should_execute),
|
||||
no_route_reason: toNullableString(item.no_route_reason ?? null),
|
||||
clarification_reason: toNullableString(item.clarification_reason ?? null)
|
||||
}));
|
||||
}
|
||||
function normalizeRuntimeAnalysisContext(value) {
|
||||
const source = toRecordObject(value);
|
||||
const source = value;
|
||||
return {
|
||||
active: Boolean(source?.active),
|
||||
as_of_date: toNullableString(source?.as_of_date),
|
||||
|
|
@ -66,7 +46,7 @@ function normalizeRuntimeAnalysisContext(value) {
|
|||
};
|
||||
}
|
||||
function normalizeBusinessScopeResolution(value) {
|
||||
const source = toRecordObject(value);
|
||||
const source = value;
|
||||
return {
|
||||
business_scope_raw: toStringArray(source?.business_scope_raw),
|
||||
business_scope_resolved: toStringArray(source?.business_scope_resolved),
|
||||
|
|
@ -75,7 +55,7 @@ function normalizeBusinessScopeResolution(value) {
|
|||
};
|
||||
}
|
||||
function normalizeAddressRuntimeMetaForDeep(value) {
|
||||
const source = toRecordObject(value);
|
||||
const source = value;
|
||||
if (!source) {
|
||||
return null;
|
||||
}
|
||||
|
|
@ -87,8 +67,8 @@ function normalizeAddressRuntimeMetaForDeep(value) {
|
|||
fallbackRuleHit: toNullableString(source.fallbackRuleHit),
|
||||
toolGateDecision: toNullableString(source.toolGateDecision),
|
||||
toolGateReason: toNullableString(source.toolGateReason),
|
||||
predecomposeContract: toRecordObject(source.predecomposeContract),
|
||||
orchestrationContract: toRecordObject(source.orchestrationContract)
|
||||
predecomposeContract: source.predecomposeContract ?? null,
|
||||
orchestrationContract: source.orchestrationContract ?? null
|
||||
};
|
||||
}
|
||||
function runAssistantDeepTurnResponseRuntime(input) {
|
||||
|
|
@ -107,21 +87,21 @@ function runAssistantDeepTurnResponseRuntime(input) {
|
|||
coverageEvaluationRequirements: input.coverageEvaluationRequirements,
|
||||
coverageReport: input.coverageReport,
|
||||
groundingCheck: input.groundingCheck,
|
||||
retrievalCalls: normalizeRecordArray(input.retrievalCalls),
|
||||
retrievalResultsRaw: Array.isArray(input.retrievalResultsRaw) ? input.retrievalResultsRaw : [],
|
||||
retrievalCalls: input.retrievalCalls,
|
||||
retrievalResultsRaw: input.retrievalResultsRaw,
|
||||
retrievalResults: input.retrievalResults,
|
||||
questionTypeClass: input.questionTypeClass,
|
||||
companyAnchors: input.companyAnchors,
|
||||
runtimeAnalysisContext: normalizeRuntimeAnalysisContext(input.runtimeAnalysisContext),
|
||||
businessScopeResolution: normalizeBusinessScopeResolution(input.businessScopeResolution),
|
||||
temporalGuard: normalizeRecord(input.temporalGuard),
|
||||
polarityAudit: normalizeRecord(input.polarityAudit),
|
||||
claimAnchorAudit: normalizeRecord(input.claimAnchorAudit),
|
||||
temporalGuard: input.temporalGuard,
|
||||
polarityAudit: input.polarityAudit,
|
||||
claimAnchorAudit: input.claimAnchorAudit,
|
||||
targetedEvidenceAudit: input.targetedEvidenceAudit,
|
||||
evidenceAdmissibilityGateAudit: input.evidenceAdmissibilityGateAudit,
|
||||
rbpLiveRouteAudit: input.rbpLiveRouteAudit ?? null,
|
||||
faLiveRouteAudit: input.faLiveRouteAudit ?? null,
|
||||
groundedAnswerEligibilityGuard: normalizeRecord(input.groundedAnswerEligibilityGuard),
|
||||
groundedAnswerEligibilityGuard: input.groundedAnswerEligibilityGuard,
|
||||
followupStateUsage: input.followupStateUsage,
|
||||
followupApplied: input.followupApplied,
|
||||
composition: input.composition,
|
||||
|
|
|
|||
|
|
@ -20,6 +20,21 @@ function buildRouteExecutorErrorRawResult(route, message) {
|
|||
errors: [message]
|
||||
};
|
||||
}
|
||||
function normalizeRawResult(value) {
|
||||
if (value === null || value === undefined) {
|
||||
return null;
|
||||
}
|
||||
if (typeof value === "string" || typeof value === "number" || typeof value === "boolean") {
|
||||
return value;
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
return value;
|
||||
}
|
||||
if (typeof value === "object") {
|
||||
return value;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
async function executeAssistantDeepTurnRetrievalPlan(input) {
|
||||
const normalizeRetrievalResultSafe = input.normalizeRetrievalResultFn ?? retrievalResultNormalizer_1.normalizeRetrievalResult;
|
||||
const retrievalCalls = [];
|
||||
|
|
@ -50,12 +65,13 @@ async function executeAssistantDeepTurnRetrievalPlan(input) {
|
|||
const raw = await input.executeRouteRuntime(planItem.route, planItem.fragment_text, {
|
||||
temporalHint: input.liveTemporalHint
|
||||
});
|
||||
const normalizedRaw = normalizeRawResult(raw);
|
||||
retrievalResultsRaw.push({
|
||||
fragment_id: planItem.fragment_id,
|
||||
route: planItem.route,
|
||||
raw_result: raw
|
||||
raw_result: normalizedRaw
|
||||
});
|
||||
retrievalResults.push(normalizeRetrievalResultSafe(planItem.fragment_id, planItem.requirement_ids, planItem.route, raw));
|
||||
retrievalResults.push(normalizeRetrievalResultSafe(planItem.fragment_id, planItem.requirement_ids, planItem.route, normalizedRaw));
|
||||
}
|
||||
catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
|
|
|
|||
|
|
@ -4,12 +4,12 @@ exports.buildAssistantTurnRuntimeDeps = buildAssistantTurnRuntimeDeps;
|
|||
function buildAssistantTurnRuntimeDeps(input) {
|
||||
return {
|
||||
...input.helpers,
|
||||
ensureSession: input.sessions.ensureSession,
|
||||
appendItem: input.sessions.appendItem,
|
||||
getSession: input.sessions.getSession,
|
||||
persistSession: input.sessionLogger.persistSession,
|
||||
setInvestigationState: input.sessions.setInvestigationState,
|
||||
normalize: input.normalizerService.normalize,
|
||||
ensureSession: (sessionId) => input.sessions.ensureSession(sessionId),
|
||||
appendItem: (sessionId, item) => input.sessions.appendItem(sessionId, item),
|
||||
getSession: (sessionId) => input.sessions.getSession(sessionId),
|
||||
persistSession: (session) => input.sessionLogger.persistSession(session),
|
||||
setInvestigationState: (sessionId, state) => input.sessions.setInvestigationState(sessionId, state),
|
||||
normalize: (payload) => input.normalizerService.normalize(payload),
|
||||
executeRouteRuntime: (route, fragmentText, options) => input.dataLayer.executeRouteRuntime(route, fragmentText, options),
|
||||
tryAddressQueryHandle: (messageUsed, options) => input.addressQueryService.tryHandle(messageUsed, options),
|
||||
chatClient: input.chatClient,
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ export interface BuildAssistantAddressOrchestrationRuntimeInput {
|
|||
effectiveAddressUserMessage: string;
|
||||
followupContext: unknown;
|
||||
llmPreDecomposeMeta: Record<string, unknown>;
|
||||
sessionItems?: unknown[];
|
||||
useMock: boolean;
|
||||
}) => Record<string, unknown>;
|
||||
buildAddressDialogContinuationContractV2: (
|
||||
|
|
@ -102,6 +103,7 @@ export async function buildAssistantAddressOrchestrationRuntime(
|
|||
effectiveAddressUserMessage: addressInputMessage,
|
||||
followupContext,
|
||||
llmPreDecomposeMeta: addressPreDecompose,
|
||||
sessionItems: input.sessionItems,
|
||||
useMock: input.useMock
|
||||
});
|
||||
const dialogContinuationContract = input.buildAddressDialogContinuationContractV2(
|
||||
|
|
|
|||
|
|
@ -1,4 +1,28 @@
|
|||
import type { AssistantDebugPayload } from "../types/assistant";
|
||||
import type {
|
||||
AssistantAddressRuntimeMetaForDeep,
|
||||
AssistantDebugPayload,
|
||||
AssistantDebugRouteRecord,
|
||||
AssistantFallbackType,
|
||||
AssistantProblemAnswerMode,
|
||||
AssistantRequirement,
|
||||
UnifiedRetrievalResult,
|
||||
FaLiveRouteAuditDebug,
|
||||
RbpLiveRouteAuditDebug
|
||||
} from "../types/assistant";
|
||||
import type { NormalizeResponsePayload, RouteHintSummary } from "../types/normalizer";
|
||||
import type { AnswerStructureV11 } from "../types/stage1Contracts";
|
||||
import type { InvestigationStateWithProblemUnits } from "../types/stage2ProblemUnits";
|
||||
import type { ClaimBoundAnchorAudit, TargetedEvidenceAcquisitionAudit } from "./assistantClaimBoundEvidence";
|
||||
import type { AssistantContractsBundleV1 } from "./assistantContractsBundleAssembler";
|
||||
import type { AssistantOutcomeClassV1 } from "./assistantOrchestrationContracts";
|
||||
import type { AssistantFollowupUsage } from "./assistantFollowupUsage";
|
||||
import type { CompanyAnchorSet } from "./companyAnchorResolver";
|
||||
import type {
|
||||
DomainPolarityGuardAudit,
|
||||
EvidenceAdmissibilityAudit,
|
||||
GroundedAnswerEligibilityAudit,
|
||||
TemporalGuardAudit
|
||||
} from "./assistantRuntimeGuards";
|
||||
|
||||
type RetrievalStatusItem = AssistantDebugPayload["retrieval_status"][number];
|
||||
|
||||
|
|
@ -6,18 +30,18 @@ export interface DeepAnalysisDebugPayloadInput {
|
|||
traceId: string;
|
||||
promptVersion: string;
|
||||
schemaVersion: string;
|
||||
fallbackType: unknown;
|
||||
routeSummary: unknown;
|
||||
fallbackType: AssistantFallbackType;
|
||||
routeSummary: RouteHintSummary | null;
|
||||
fragments: unknown[];
|
||||
requirementsExtracted: unknown[];
|
||||
coverageReport: unknown;
|
||||
routes: Array<Record<string, unknown>>;
|
||||
requirementsExtracted: AssistantRequirement[];
|
||||
coverageReport: AssistantDebugPayload["coverage_report"];
|
||||
routes: AssistantDebugRouteRecord[];
|
||||
retrievalStatus: RetrievalStatusItem[];
|
||||
retrievalResults: unknown[];
|
||||
groundingCheck: unknown;
|
||||
retrievalResults: UnifiedRetrievalResult[];
|
||||
groundingCheck: AssistantDebugPayload["answer_grounding_check"];
|
||||
droppedIntentSegments: string[];
|
||||
questionTypeClass: string;
|
||||
companyAnchors: unknown;
|
||||
companyAnchors: CompanyAnchorSet | null;
|
||||
runtimeAnalysisContext: {
|
||||
active: boolean;
|
||||
as_of_date: string | null;
|
||||
|
|
@ -32,40 +56,27 @@ export interface DeepAnalysisDebugPayloadInput {
|
|||
company_grounding_applied?: boolean;
|
||||
scope_resolution_reason?: string[];
|
||||
};
|
||||
temporalGuard: Record<string, unknown>;
|
||||
polarityAudit: Record<string, unknown>;
|
||||
claimAnchorAudit: Record<string, unknown>;
|
||||
targetedEvidenceAudit: unknown;
|
||||
evidenceAdmissibilityGateAudit: unknown;
|
||||
rbpLiveRouteAudit: unknown | null;
|
||||
faLiveRouteAudit: unknown | null;
|
||||
groundedAnswerEligibilityGuard: Record<string, unknown>;
|
||||
followupStateUsage: unknown | null;
|
||||
temporalGuard: TemporalGuardAudit;
|
||||
polarityAudit: DomainPolarityGuardAudit;
|
||||
claimAnchorAudit: ClaimBoundAnchorAudit;
|
||||
targetedEvidenceAudit: TargetedEvidenceAcquisitionAudit;
|
||||
evidenceAdmissibilityGateAudit: EvidenceAdmissibilityAudit;
|
||||
rbpLiveRouteAudit: RbpLiveRouteAuditDebug | null;
|
||||
faLiveRouteAudit: FaLiveRouteAuditDebug | null;
|
||||
groundedAnswerEligibilityGuard: GroundedAnswerEligibilityAudit;
|
||||
followupStateUsage: AssistantFollowupUsage | null;
|
||||
compositionDebug: {
|
||||
problem_centric_answer_applied?: boolean;
|
||||
problem_units_used_count?: number;
|
||||
problem_answer_mode?: string;
|
||||
problem_answer_mode?: AssistantProblemAnswerMode;
|
||||
problem_unit_ids_used?: string[];
|
||||
};
|
||||
addressRuntimeMetaForDeep:
|
||||
| {
|
||||
attempted?: boolean;
|
||||
applied?: boolean;
|
||||
reason?: string | null;
|
||||
provider?: string | null;
|
||||
fallbackRuleHit?: string | null;
|
||||
toolGateDecision?: string | null;
|
||||
toolGateReason?: string | null;
|
||||
predecomposeContract?: unknown;
|
||||
orchestrationContract?: unknown;
|
||||
}
|
||||
| null
|
||||
| undefined;
|
||||
outcomeClassV1: unknown;
|
||||
assistantOrchestrationContractsV1: unknown;
|
||||
answerStructureV11: unknown;
|
||||
investigationStateSnapshot: unknown;
|
||||
normalizedPayload: unknown;
|
||||
addressRuntimeMetaForDeep: AssistantAddressRuntimeMetaForDeep | null | undefined;
|
||||
outcomeClassV1: AssistantOutcomeClassV1;
|
||||
assistantOrchestrationContractsV1: AssistantContractsBundleV1["assistantOrchestrationContractsV1"];
|
||||
answerStructureV11: AnswerStructureV11 | null;
|
||||
investigationStateSnapshot: InvestigationStateWithProblemUnits | null;
|
||||
normalizedPayload: NormalizeResponsePayload["normalized"];
|
||||
}
|
||||
|
||||
function toAnalysisContext(input: DeepAnalysisDebugPayloadInput["runtimeAnalysisContext"]): Record<string, unknown> | null {
|
||||
|
|
|
|||
|
|
@ -49,14 +49,7 @@ const KNOWN_GUARD_DOMAINS = [
|
|||
"fixed_asset_amortization"
|
||||
] as const;
|
||||
|
||||
function toRecordObject(value: unknown): Record<string, unknown> | null {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) {
|
||||
return null;
|
||||
}
|
||||
return value as Record<string, unknown>;
|
||||
}
|
||||
|
||||
function toStringArray(value: unknown): string[] {
|
||||
function toStringArray(value: string[] | null | undefined): string[] {
|
||||
if (!Array.isArray(value)) {
|
||||
return [];
|
||||
}
|
||||
|
|
@ -65,11 +58,11 @@ function toStringArray(value: unknown): string[] {
|
|||
.filter((item) => item.length > 0);
|
||||
}
|
||||
|
||||
function toCompanyAnchorSet(value: unknown): CompanyAnchorSet | null {
|
||||
const source = toRecordObject(value);
|
||||
if (!source) {
|
||||
function toCompanyAnchorSet(value: Partial<CompanyAnchorSet> | null | undefined): CompanyAnchorSet | null {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) {
|
||||
return null;
|
||||
}
|
||||
const source: Partial<CompanyAnchorSet> = value;
|
||||
return {
|
||||
contract_numbers: toStringArray(source.contract_numbers),
|
||||
document_numbers: toStringArray(source.document_numbers),
|
||||
|
|
@ -82,14 +75,13 @@ function toCompanyAnchorSet(value: unknown): CompanyAnchorSet | null {
|
|||
};
|
||||
}
|
||||
|
||||
function toClaimBoundPrimaryPeriod(value: unknown): ClaimBoundPrimaryPeriod {
|
||||
const source = toRecordObject(value);
|
||||
if (!source) {
|
||||
function toClaimBoundPrimaryPeriod(value: ClaimBoundPrimaryPeriod | null | undefined): ClaimBoundPrimaryPeriod {
|
||||
if (!value || typeof value !== "object") {
|
||||
return null;
|
||||
}
|
||||
const from = typeof source.from === "string" ? source.from.trim() : "";
|
||||
const to = typeof source.to === "string" ? source.to.trim() : "";
|
||||
const granularity = source.granularity === "day" || source.granularity === "month" ? source.granularity : null;
|
||||
const from = typeof value.from === "string" ? value.from.trim() : "";
|
||||
const to = typeof value.to === "string" ? value.to.trim() : "";
|
||||
const granularity = value.granularity === "day" || value.granularity === "month" ? value.granularity : null;
|
||||
if (!from || !to || !granularity) {
|
||||
return null;
|
||||
}
|
||||
|
|
@ -101,7 +93,7 @@ function toClaimBoundPrimaryPeriod(value: unknown): ClaimBoundPrimaryPeriod {
|
|||
}
|
||||
|
||||
function normalizeFocusDomainForGuards(
|
||||
value: unknown
|
||||
value: string | null | undefined
|
||||
): AssistantDeepTurnRetrievalGuardPipelineInput["focusDomainForGuards"] {
|
||||
const normalized = typeof value === "string" ? value.trim() : "";
|
||||
if (!normalized) {
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ import type {
|
|||
import type { RunAssistantDeepTurnAnalysisAttemptRuntimeInput } from "./assistantDeepTurnAnalysisAttemptRuntimeAdapter";
|
||||
import type { RunAssistantDeepTurnAnalysisRuntimeOutput } from "./assistantDeepTurnAnalysisRuntimeAdapter";
|
||||
import type { RunAssistantDeepTurnResponseAttemptRuntimeInput } from "./assistantDeepTurnResponseAttemptRuntimeAdapter";
|
||||
import type { AssistantMessageResponsePayload } from "../types/assistant";
|
||||
import { isAssistantFollowupApplied } from "./assistantFollowupUsage";
|
||||
|
||||
export interface BuildAssistantDeepTurnNormalizationRuntimeInputInput {
|
||||
|
|
@ -88,7 +89,7 @@ export function buildAssistantDeepTurnAnalysisAttemptRuntimeInput(
|
|||
};
|
||||
}
|
||||
|
||||
export interface BuildAssistantDeepTurnResponseAttemptRuntimeInputInput<ResponseType = unknown> {
|
||||
export interface BuildAssistantDeepTurnResponseAttemptRuntimeInputInput<ResponseType = AssistantMessageResponsePayload> {
|
||||
featureInvestigationStateV1: boolean;
|
||||
featureContractsV11: boolean;
|
||||
featureAnswerPolicyV11: boolean;
|
||||
|
|
@ -115,7 +116,7 @@ export interface BuildAssistantDeepTurnResponseAttemptRuntimeInputInput<Response
|
|||
deepTurnAnalysisRuntime: RunAssistantDeepTurnAnalysisRuntimeOutput;
|
||||
}
|
||||
|
||||
export function buildAssistantDeepTurnResponseAttemptRuntimeInput<ResponseType = unknown>(
|
||||
export function buildAssistantDeepTurnResponseAttemptRuntimeInput<ResponseType = AssistantMessageResponsePayload>(
|
||||
input: BuildAssistantDeepTurnResponseAttemptRuntimeInputInput<ResponseType>
|
||||
): RunAssistantDeepTurnResponseAttemptRuntimeInput<ResponseType> {
|
||||
return {
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ export interface BuildAssistantDeepTurnCompositionInput {
|
|||
groundingCheck: AnswerGroundingCheck;
|
||||
followupUsage: AssistantFollowupUsage | null | undefined;
|
||||
investigationState: InvestigationStateWithProblemUnits | null | undefined;
|
||||
companyAnchors: unknown;
|
||||
companyAnchors: CompanyAnchorSet | null;
|
||||
normalizedPayload: NormalizeResponsePayload["normalized"];
|
||||
featureAnswerPolicyV11: boolean;
|
||||
featureProblemCentricAnswerV1: boolean;
|
||||
|
|
@ -34,7 +34,7 @@ export interface AssistantDeepTurnCompositionOutput {
|
|||
composition: ReturnType<typeof composeAssistantAnswer>;
|
||||
}
|
||||
|
||||
function toStringArray(value: unknown): string[] {
|
||||
function toStringArray(value: string[] | null | undefined): string[] {
|
||||
if (!Array.isArray(value)) {
|
||||
return [];
|
||||
}
|
||||
|
|
@ -43,11 +43,11 @@ function toStringArray(value: unknown): string[] {
|
|||
.filter((item) => item.length > 0);
|
||||
}
|
||||
|
||||
function normalizeCompanyAnchorSet(value: unknown): CompanyAnchorSet | null {
|
||||
function normalizeCompanyAnchorSet(value: Partial<CompanyAnchorSet> | null | undefined): CompanyAnchorSet | null {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) {
|
||||
return null;
|
||||
}
|
||||
const source = value as Record<string, unknown>;
|
||||
const source: Partial<CompanyAnchorSet> = value;
|
||||
return {
|
||||
contract_numbers: toStringArray(source.contract_numbers),
|
||||
document_numbers: toStringArray(source.document_numbers),
|
||||
|
|
|
|||
|
|
@ -61,14 +61,11 @@ export interface BuildAssistantDeepTurnRuntimeContextInput {
|
|||
userMessage: string;
|
||||
companyAnchors: CompanyAnchorSet;
|
||||
focusDomainHint: string | null;
|
||||
primaryPeriod: unknown;
|
||||
primaryPeriod: ClaimBoundAnchorAudit["primary_period"] | null;
|
||||
}) => ClaimBoundAnchorAudit;
|
||||
resolveBusinessScopeFromLiveContext: (input: {
|
||||
current: {
|
||||
route_summary_resolved: RouteHintSummary | null;
|
||||
[key: string]: unknown;
|
||||
};
|
||||
temporalGuard: unknown;
|
||||
current: AssistantBusinessScopeResolution;
|
||||
temporalGuard: TemporalGuardAudit;
|
||||
claimType: string;
|
||||
focusDomainHint: string | null;
|
||||
userMessage: string;
|
||||
|
|
@ -83,7 +80,6 @@ export interface AssistantBusinessScopeResolution {
|
|||
business_scope_resolved?: string[];
|
||||
company_grounding_applied?: boolean;
|
||||
scope_resolution_reason?: string[];
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
export interface BuildAssistantDeepTurnRuntimeContextOutput {
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
import type { AssistantConversationItem, AssistantDebugPayload, AssistantMessageResponsePayload, AssistantReplyType } from "../types/assistant";
|
||||
import type { DeepAnalysisLogDetails } from "./assistantMessageLogAssembler";
|
||||
import { buildAssistantDeepTurnSuccessResponse } from "./assistantDeepTurnResponseBuilder";
|
||||
import type { CommitAssistantTurnAndLogOutput } from "./assistantTurnCommitRuntimeAdapter";
|
||||
import { commitAssistantTurnAndLog } from "./assistantTurnCommitRuntimeAdapter";
|
||||
|
|
@ -9,7 +10,7 @@ export interface FinalizeAssistantDeepTurnInput {
|
|||
replyType: AssistantReplyType;
|
||||
assistantItem: AssistantConversationItem;
|
||||
debug: AssistantDebugPayload;
|
||||
deepAnalysisLogDetails: Record<string, unknown>;
|
||||
deepAnalysisLogDetails: DeepAnalysisLogDetails;
|
||||
appendItem: Parameters<typeof commitAssistantTurnAndLog>[0]["appendItem"];
|
||||
getSession: Parameters<typeof commitAssistantTurnAndLog>[0]["getSession"];
|
||||
persistSession: Parameters<typeof commitAssistantTurnAndLog>[0]["persistSession"];
|
||||
|
|
|
|||
|
|
@ -1,8 +1,34 @@
|
|||
import type { AssistantReplyType, AssistantRequirement, AnswerGroundingCheck, RequirementCoverageReport, UnifiedRetrievalResult } from "../types/assistant";
|
||||
import type {
|
||||
AssistantAddressRuntimeMetaForDeep,
|
||||
AssistantDebugRouteRecord,
|
||||
AssistantExecutionStateRecord,
|
||||
AssistantFallbackType,
|
||||
AssistantProblemAnswerMode,
|
||||
AssistantReplyType,
|
||||
AssistantRequirement,
|
||||
AnswerGroundingCheck,
|
||||
FaLiveRouteAuditDebug,
|
||||
RbpLiveRouteAuditDebug,
|
||||
RequirementCoverageReport,
|
||||
UnifiedRetrievalResult
|
||||
} from "../types/assistant";
|
||||
import type { NormalizeResponsePayload, RouteHintSummary } from "../types/normalizer";
|
||||
import type { AnswerStructureV11 } from "../types/stage1Contracts";
|
||||
import type { InvestigationStateWithProblemUnits } from "../types/stage2ProblemUnits";
|
||||
import type { AssistantDeepTurnPackagingInput } from "./assistantDeepTurnPackaging";
|
||||
import type { AssistantFollowupUsage } from "./assistantFollowupUsage";
|
||||
import type { ClaimBoundAnchorAudit, TargetedEvidenceAcquisitionAudit } from "./assistantClaimBoundEvidence";
|
||||
import type { CompanyAnchorSet } from "./companyAnchorResolver";
|
||||
import type {
|
||||
DomainPolarityGuardAudit,
|
||||
EvidenceAdmissibilityAudit,
|
||||
GroundedAnswerEligibilityAudit,
|
||||
TemporalGuardAudit
|
||||
} from "./assistantRuntimeGuards";
|
||||
import type {
|
||||
AssistantRetrievalCallRecord,
|
||||
AssistantRetrievalRawResultRecord
|
||||
} from "./assistantDeepTurnRetrievalRuntimeAdapter";
|
||||
|
||||
export interface AssistantDeepTurnInputBuilderArgs {
|
||||
sessionId: string;
|
||||
|
|
@ -36,13 +62,13 @@ export interface AssistantDeepTurnInputBuilderArgs {
|
|||
coverageEvaluationRequirements: AssistantRequirement[];
|
||||
coverageReport: RequirementCoverageReport;
|
||||
groundingCheck: AnswerGroundingCheck;
|
||||
retrievalCalls: Array<Record<string, unknown>>;
|
||||
retrievalResultsRaw: unknown[];
|
||||
retrievalCalls: AssistantRetrievalCallRecord[];
|
||||
retrievalResultsRaw: AssistantRetrievalRawResultRecord[];
|
||||
retrievalResults: UnifiedRetrievalResult[];
|
||||
routesForDebug: Array<Record<string, unknown>>;
|
||||
resolvedExecutionState: unknown;
|
||||
routesForDebug: AssistantDebugRouteRecord[];
|
||||
resolvedExecutionState: AssistantExecutionStateRecord[];
|
||||
questionTypeClass: string;
|
||||
companyAnchors: unknown;
|
||||
companyAnchors: CompanyAnchorSet | null;
|
||||
runtimeAnalysisContext: {
|
||||
active: boolean;
|
||||
as_of_date: string | null;
|
||||
|
|
@ -57,42 +83,29 @@ export interface AssistantDeepTurnInputBuilderArgs {
|
|||
company_grounding_applied?: boolean;
|
||||
scope_resolution_reason?: string[];
|
||||
};
|
||||
temporalGuard: Record<string, unknown>;
|
||||
polarityAudit: Record<string, unknown>;
|
||||
claimAnchorAudit: Record<string, unknown>;
|
||||
targetedEvidenceAudit: unknown;
|
||||
evidenceAdmissibilityGateAudit: unknown;
|
||||
rbpLiveRouteAudit: unknown | null;
|
||||
faLiveRouteAudit: unknown | null;
|
||||
groundedAnswerEligibilityGuard: Record<string, unknown>;
|
||||
temporalGuard: TemporalGuardAudit;
|
||||
polarityAudit: DomainPolarityGuardAudit;
|
||||
claimAnchorAudit: ClaimBoundAnchorAudit;
|
||||
targetedEvidenceAudit: TargetedEvidenceAcquisitionAudit;
|
||||
evidenceAdmissibilityGateAudit: EvidenceAdmissibilityAudit;
|
||||
rbpLiveRouteAudit: RbpLiveRouteAuditDebug | null;
|
||||
faLiveRouteAudit: FaLiveRouteAuditDebug | null;
|
||||
groundedAnswerEligibilityGuard: GroundedAnswerEligibilityAudit;
|
||||
followupStateUsage?: AssistantFollowupUsage | null;
|
||||
composition: {
|
||||
reply_type: AssistantReplyType;
|
||||
fallback_type: unknown;
|
||||
fallback_type: AssistantFallbackType;
|
||||
answer_structure_v11?: AnswerStructureV11 | null;
|
||||
problem_centric_answer_applied?: boolean;
|
||||
problem_units_used_count?: number;
|
||||
problem_answer_mode?: string;
|
||||
problem_unit_ids_used?: unknown;
|
||||
problem_answer_mode?: AssistantProblemAnswerMode;
|
||||
problem_unit_ids_used?: string[];
|
||||
};
|
||||
safeAssistantReplyBase: string;
|
||||
featureContractsV11: boolean;
|
||||
featureAnswerPolicyV11: boolean;
|
||||
investigationStateSnapshot: unknown;
|
||||
addressRuntimeMetaForDeep:
|
||||
| {
|
||||
attempted?: boolean;
|
||||
applied?: boolean;
|
||||
reason?: string | null;
|
||||
provider?: string | null;
|
||||
fallbackRuleHit?: string | null;
|
||||
toolGateDecision?: string | null;
|
||||
toolGateReason?: string | null;
|
||||
predecomposeContract?: unknown;
|
||||
orchestrationContract?: unknown;
|
||||
}
|
||||
| null
|
||||
| undefined;
|
||||
investigationStateSnapshot: InvestigationStateWithProblemUnits | null;
|
||||
addressRuntimeMetaForDeep: AssistantAddressRuntimeMetaForDeep | null | undefined;
|
||||
}
|
||||
|
||||
export function buildAssistantDeepTurnPackagingInput(args: AssistantDeepTurnInputBuilderArgs): AssistantDeepTurnPackagingInput {
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
import type { AssistantMessageRequestPayload } from "../types/assistant";
|
||||
import type { NormalizeRequestPayload, NormalizeResponsePayload } from "../types/normalizer";
|
||||
import type { InvestigationStateWithProblemUnits } from "../types/stage2ProblemUnits";
|
||||
import type { AssistantFollowupUsage } from "./assistantFollowupUsage";
|
||||
|
||||
export interface AssistantDeepTurnFollowupBinding {
|
||||
|
|
@ -13,11 +14,11 @@ export interface BuildAssistantDeepTurnNormalizationRuntimeInput {
|
|||
payload: AssistantMessageRequestPayload;
|
||||
featureInvestigationStateV1: boolean;
|
||||
featureStateFollowupBindingV1: boolean;
|
||||
sessionInvestigationState: unknown | null | undefined;
|
||||
sessionInvestigationState: InvestigationStateWithProblemUnits | null | undefined;
|
||||
buildFollowupStateBinding: (input: {
|
||||
userMessage: string;
|
||||
payloadContext: NormalizeRequestPayload["context"] | undefined;
|
||||
investigationState: unknown;
|
||||
investigationState: InvestigationStateWithProblemUnits;
|
||||
}) => AssistantDeepTurnFollowupBinding;
|
||||
normalize: (payload: NormalizeRequestPayload) => Promise<NormalizeResponsePayload>;
|
||||
}
|
||||
|
|
@ -31,14 +32,18 @@ export interface BuildAssistantDeepTurnNormalizationRuntimeOutput {
|
|||
export async function buildAssistantDeepTurnNormalizationRuntime(
|
||||
input: BuildAssistantDeepTurnNormalizationRuntimeInput
|
||||
): Promise<BuildAssistantDeepTurnNormalizationRuntimeOutput> {
|
||||
const followupBinding =
|
||||
const investigationState = input.sessionInvestigationState;
|
||||
const canUseFollowupBinding =
|
||||
input.featureInvestigationStateV1 &&
|
||||
input.featureStateFollowupBindingV1 &&
|
||||
Boolean(input.sessionInvestigationState)
|
||||
investigationState !== null &&
|
||||
investigationState !== undefined;
|
||||
const followupBinding =
|
||||
canUseFollowupBinding
|
||||
? input.buildFollowupStateBinding({
|
||||
userMessage: input.userMessage,
|
||||
payloadContext: input.payload.context,
|
||||
investigationState: input.sessionInvestigationState as unknown
|
||||
investigationState
|
||||
})
|
||||
: {
|
||||
normalizedQuestion: input.userMessage,
|
||||
|
|
|
|||
|
|
@ -1,14 +1,28 @@
|
|||
import type {
|
||||
AssistantAddressRuntimeMetaForDeep,
|
||||
AssistantConversationItem,
|
||||
AssistantDebugPayload,
|
||||
AssistantDebugRouteRecord,
|
||||
AssistantExecutionStateRecord,
|
||||
AssistantFallbackType,
|
||||
AssistantProblemAnswerMode,
|
||||
AssistantReplyType,
|
||||
AssistantRequirement,
|
||||
AnswerGroundingCheck,
|
||||
FaLiveRouteAuditDebug,
|
||||
RbpLiveRouteAuditDebug,
|
||||
RequirementCoverageReport,
|
||||
UnifiedRetrievalResult
|
||||
} from "../types/assistant";
|
||||
import type { NormalizeResponsePayload, RouteHintSummary } from "../types/normalizer";
|
||||
import type {
|
||||
NormalizeResponsePayload,
|
||||
NormalizedQueryV2,
|
||||
NormalizedQueryV2_0_1,
|
||||
NormalizedQueryV2_0_2,
|
||||
RouteHintSummary
|
||||
} from "../types/normalizer";
|
||||
import type { AnswerStructureV11 } from "../types/stage1Contracts";
|
||||
import type { InvestigationStateWithProblemUnits } from "../types/stage2ProblemUnits";
|
||||
import {
|
||||
assembleAssistantEvidenceBundle,
|
||||
type AssistantEvidenceBundleAssembly
|
||||
|
|
@ -16,8 +30,23 @@ import {
|
|||
import { assembleAssistantContractsBundleV1, type AssistantContractsBundleV1 } from "./assistantContractsBundleAssembler";
|
||||
import { buildDeepAnswerArtifacts, buildAssistantConversationItem, type DeepAnswerArtifacts } from "./assistantDeepResponseAssembler";
|
||||
import { buildDeepAnalysisDebugPayload } from "./assistantDebugPayloadAssembler";
|
||||
import { buildDeepAnalysisProcessedLogDetails } from "./assistantMessageLogAssembler";
|
||||
import {
|
||||
buildDeepAnalysisProcessedLogDetails,
|
||||
type DeepAnalysisLogDetails
|
||||
} from "./assistantMessageLogAssembler";
|
||||
import type { AssistantFollowupUsage } from "./assistantFollowupUsage";
|
||||
import type { ClaimBoundAnchorAudit, TargetedEvidenceAcquisitionAudit } from "./assistantClaimBoundEvidence";
|
||||
import type { CompanyAnchorSet } from "./companyAnchorResolver";
|
||||
import type {
|
||||
DomainPolarityGuardAudit,
|
||||
EvidenceAdmissibilityAudit,
|
||||
GroundedAnswerEligibilityAudit,
|
||||
TemporalGuardAudit
|
||||
} from "./assistantRuntimeGuards";
|
||||
import type {
|
||||
AssistantRetrievalCallRecord,
|
||||
AssistantRetrievalRawResultRecord
|
||||
} from "./assistantDeepTurnRetrievalRuntimeAdapter";
|
||||
|
||||
export interface AssistantDeepTurnPackagingInput {
|
||||
sessionId: string;
|
||||
|
|
@ -51,13 +80,13 @@ export interface AssistantDeepTurnPackagingInput {
|
|||
coverageEvaluationRequirements: AssistantRequirement[];
|
||||
coverageReport: RequirementCoverageReport;
|
||||
groundingCheck: AnswerGroundingCheck;
|
||||
retrievalCalls: Array<Record<string, unknown>>;
|
||||
retrievalResultsRaw: unknown[];
|
||||
retrievalCalls: AssistantRetrievalCallRecord[];
|
||||
retrievalResultsRaw: AssistantRetrievalRawResultRecord[];
|
||||
retrievalResults: UnifiedRetrievalResult[];
|
||||
routesForDebug: Array<Record<string, unknown>>;
|
||||
resolvedExecutionState: unknown;
|
||||
routesForDebug: AssistantDebugRouteRecord[];
|
||||
resolvedExecutionState: AssistantExecutionStateRecord[];
|
||||
questionTypeClass: string;
|
||||
companyAnchors: unknown;
|
||||
companyAnchors: CompanyAnchorSet | null;
|
||||
runtimeAnalysisContext: {
|
||||
active: boolean;
|
||||
as_of_date: string | null;
|
||||
|
|
@ -72,42 +101,29 @@ export interface AssistantDeepTurnPackagingInput {
|
|||
company_grounding_applied?: boolean;
|
||||
scope_resolution_reason?: string[];
|
||||
};
|
||||
temporalGuard: Record<string, unknown>;
|
||||
polarityAudit: Record<string, unknown>;
|
||||
claimAnchorAudit: Record<string, unknown>;
|
||||
targetedEvidenceAudit: unknown;
|
||||
evidenceAdmissibilityGateAudit: unknown;
|
||||
rbpLiveRouteAudit: unknown | null;
|
||||
faLiveRouteAudit: unknown | null;
|
||||
groundedAnswerEligibilityGuard: Record<string, unknown>;
|
||||
temporalGuard: TemporalGuardAudit;
|
||||
polarityAudit: DomainPolarityGuardAudit;
|
||||
claimAnchorAudit: ClaimBoundAnchorAudit;
|
||||
targetedEvidenceAudit: TargetedEvidenceAcquisitionAudit;
|
||||
evidenceAdmissibilityGateAudit: EvidenceAdmissibilityAudit;
|
||||
rbpLiveRouteAudit: RbpLiveRouteAuditDebug | null;
|
||||
faLiveRouteAudit: FaLiveRouteAuditDebug | null;
|
||||
groundedAnswerEligibilityGuard: GroundedAnswerEligibilityAudit;
|
||||
followupStateUsage: AssistantFollowupUsage | null;
|
||||
composition: {
|
||||
reply_type: AssistantReplyType;
|
||||
fallback_type: unknown;
|
||||
fallback_type: AssistantFallbackType;
|
||||
answer_structure_v11?: AnswerStructureV11 | null;
|
||||
problem_centric_answer_applied?: boolean;
|
||||
problem_units_used_count?: number;
|
||||
problem_answer_mode?: string;
|
||||
problem_answer_mode?: AssistantProblemAnswerMode;
|
||||
problem_unit_ids_used?: string[];
|
||||
};
|
||||
safeAssistantReplyBase: string;
|
||||
featureContractsV11: boolean;
|
||||
featureAnswerPolicyV11: boolean;
|
||||
investigationStateSnapshot: unknown;
|
||||
addressRuntimeMetaForDeep:
|
||||
| {
|
||||
attempted?: boolean;
|
||||
applied?: boolean;
|
||||
reason?: string | null;
|
||||
provider?: string | null;
|
||||
fallbackRuleHit?: string | null;
|
||||
toolGateDecision?: string | null;
|
||||
toolGateReason?: string | null;
|
||||
predecomposeContract?: unknown;
|
||||
orchestrationContract?: unknown;
|
||||
}
|
||||
| null
|
||||
| undefined;
|
||||
investigationStateSnapshot: InvestigationStateWithProblemUnits | null;
|
||||
addressRuntimeMetaForDeep: AssistantAddressRuntimeMetaForDeep | null | undefined;
|
||||
}
|
||||
|
||||
export interface AssistantDeepTurnPackagingOutput {
|
||||
|
|
@ -116,12 +132,25 @@ export interface AssistantDeepTurnPackagingOutput {
|
|||
deepAnswerArtifacts: DeepAnswerArtifacts;
|
||||
debug: AssistantDebugPayload;
|
||||
assistantItem: AssistantConversationItem;
|
||||
deepAnalysisLogDetails: Record<string, unknown>;
|
||||
deepAnalysisLogDetails: DeepAnalysisLogDetails;
|
||||
}
|
||||
|
||||
type NormalizedFragments =
|
||||
| NormalizedQueryV2["fragments"]
|
||||
| NormalizedQueryV2_0_1["fragments"]
|
||||
| NormalizedQueryV2_0_2["fragments"];
|
||||
|
||||
function extractNormalizedFragments(normalized: NormalizeResponsePayload["normalized"] | null): NormalizedFragments {
|
||||
if (!normalized || typeof normalized !== "object") {
|
||||
return [];
|
||||
}
|
||||
const source = normalized as NormalizedQueryV2 | NormalizedQueryV2_0_1 | NormalizedQueryV2_0_2;
|
||||
return Array.isArray(source.fragments) ? source.fragments : [];
|
||||
}
|
||||
|
||||
export function assembleAssistantDeepTurnPackaging(input: AssistantDeepTurnPackagingInput): AssistantDeepTurnPackagingOutput {
|
||||
const normalizedPayload = (input.normalized.normalized ?? null) as Record<string, unknown> | null;
|
||||
const normalizedFragments = Array.isArray(normalizedPayload?.["fragments"]) ? (normalizedPayload?.["fragments"] as unknown[]) : [];
|
||||
const normalizedPayload = input.normalized.normalized ?? null;
|
||||
const normalizedFragments = extractNormalizedFragments(normalizedPayload);
|
||||
const evidenceBundleAssembly = assembleAssistantEvidenceBundle({
|
||||
retrievalCalls: input.retrievalCalls,
|
||||
retrievalResults: input.retrievalResults
|
||||
|
|
@ -232,7 +261,7 @@ export function assembleAssistantDeepTurnPackaging(input: AssistantDeepTurnPacka
|
|||
problem_units_used_count: input.composition.problem_units_used_count ?? 0,
|
||||
problem_answer_mode: input.composition.problem_answer_mode ?? "stage1_policy_v11",
|
||||
problem_unit_ids_used: Array.isArray(input.composition.problem_unit_ids_used) ? input.composition.problem_unit_ids_used : [],
|
||||
fallback_type: input.composition.fallback_type as string
|
||||
fallback_type: input.composition.fallback_type
|
||||
},
|
||||
outcomeClassV1: contractsBundleV1.outcomeClassV1,
|
||||
assistantOrchestrationContractsV1: contractsBundleV1.assistantOrchestrationContractsV1,
|
||||
|
|
|
|||
|
|
@ -1,9 +1,14 @@
|
|||
import { nanoid } from "nanoid";
|
||||
import type {
|
||||
AssistantAddressRuntimeMetaForDeep,
|
||||
AssistantConversationItem,
|
||||
AssistantDebugPayload,
|
||||
AssistantDebugRouteRecord,
|
||||
AssistantExecutionStateRecord,
|
||||
AssistantRequirement,
|
||||
AnswerGroundingCheck,
|
||||
FaLiveRouteAuditDebug,
|
||||
RbpLiveRouteAuditDebug,
|
||||
RequirementCoverageReport,
|
||||
UnifiedRetrievalResult
|
||||
} from "../types/assistant";
|
||||
|
|
@ -13,6 +18,18 @@ import type { AssistantDeepTurnInputBuilderArgs } from "./assistantDeepTurnInput
|
|||
import { buildAssistantDeepTurnPackagingInput } from "./assistantDeepTurnInputBuilder";
|
||||
import { assembleAssistantDeepTurnPackaging } from "./assistantDeepTurnPackaging";
|
||||
import type { AssistantFollowupUsage } from "./assistantFollowupUsage";
|
||||
import type { ClaimBoundAnchorAudit, TargetedEvidenceAcquisitionAudit } from "./assistantClaimBoundEvidence";
|
||||
import type { CompanyAnchorSet } from "./companyAnchorResolver";
|
||||
import type {
|
||||
DomainPolarityGuardAudit,
|
||||
EvidenceAdmissibilityAudit,
|
||||
GroundedAnswerEligibilityAudit,
|
||||
TemporalGuardAudit
|
||||
} from "./assistantRuntimeGuards";
|
||||
import type {
|
||||
AssistantRetrievalCallRecord,
|
||||
AssistantRetrievalRawResultRecord
|
||||
} from "./assistantDeepTurnRetrievalRuntimeAdapter";
|
||||
import type {
|
||||
AssistantAnalysisContextForContract,
|
||||
AssistantRuntimeAnalysisContextForPrePackaging
|
||||
|
|
@ -22,6 +39,7 @@ import {
|
|||
buildAssistantInvestigationStateSnapshot,
|
||||
persistAssistantInvestigationStateSnapshot
|
||||
} from "./assistantInvestigationStateRuntimeAdapter";
|
||||
import type { DeepAnalysisLogDetails } from "./assistantMessageLogAssembler";
|
||||
|
||||
type AssistantDeepTurnCompositionForPackaging = AssistantDeepTurnInputBuilderArgs["composition"] & {
|
||||
assistant_reply: string;
|
||||
|
|
@ -45,44 +63,31 @@ export interface AssistantDeepTurnPackagingRuntimeInput {
|
|||
coverageEvaluationRequirements: AssistantRequirement[];
|
||||
coverageReport: RequirementCoverageReport;
|
||||
groundingCheck: AnswerGroundingCheck;
|
||||
retrievalCalls: Array<Record<string, unknown>>;
|
||||
retrievalResultsRaw: unknown[];
|
||||
retrievalCalls: AssistantRetrievalCallRecord[];
|
||||
retrievalResultsRaw: AssistantRetrievalRawResultRecord[];
|
||||
retrievalResults: UnifiedRetrievalResult[];
|
||||
questionTypeClass: string;
|
||||
companyAnchors: unknown;
|
||||
companyAnchors: CompanyAnchorSet | null;
|
||||
runtimeAnalysisContext: AssistantDeepTurnInputBuilderArgs["runtimeAnalysisContext"];
|
||||
businessScopeResolution: AssistantDeepTurnInputBuilderArgs["businessScopeResolution"];
|
||||
temporalGuard: Record<string, unknown>;
|
||||
polarityAudit: Record<string, unknown>;
|
||||
claimAnchorAudit: Record<string, unknown>;
|
||||
targetedEvidenceAudit: unknown;
|
||||
evidenceAdmissibilityGateAudit: unknown;
|
||||
rbpLiveRouteAudit: unknown | null;
|
||||
faLiveRouteAudit: unknown | null;
|
||||
groundedAnswerEligibilityGuard: Record<string, unknown>;
|
||||
temporalGuard: TemporalGuardAudit;
|
||||
polarityAudit: DomainPolarityGuardAudit;
|
||||
claimAnchorAudit: ClaimBoundAnchorAudit;
|
||||
targetedEvidenceAudit: TargetedEvidenceAcquisitionAudit;
|
||||
evidenceAdmissibilityGateAudit: EvidenceAdmissibilityAudit;
|
||||
rbpLiveRouteAudit: RbpLiveRouteAuditDebug | null;
|
||||
faLiveRouteAudit: FaLiveRouteAuditDebug | null;
|
||||
groundedAnswerEligibilityGuard: GroundedAnswerEligibilityAudit;
|
||||
followupStateUsage?: AssistantFollowupUsage | null;
|
||||
followupApplied: boolean;
|
||||
composition: AssistantDeepTurnCompositionForPackaging;
|
||||
featureContractsV11: boolean;
|
||||
featureAnswerPolicyV11: boolean;
|
||||
previousInvestigationState: InvestigationStateWithProblemUnits | null | undefined;
|
||||
addressRuntimeMetaForDeep:
|
||||
| {
|
||||
attempted?: boolean;
|
||||
applied?: boolean;
|
||||
reason?: string | null;
|
||||
provider?: string | null;
|
||||
fallbackRuleHit?: string | null;
|
||||
toolGateDecision?: string | null;
|
||||
toolGateReason?: string | null;
|
||||
predecomposeContract?: unknown;
|
||||
orchestrationContract?: unknown;
|
||||
}
|
||||
| null
|
||||
| undefined;
|
||||
addressRuntimeMetaForDeep: AssistantAddressRuntimeMetaForDeep | null | undefined;
|
||||
extractDroppedIntentSegments: (normalizedPayload: NormalizeResponsePayload["normalized"]) => string[];
|
||||
buildDebugRoutes: (routeSummary: RouteHintSummary | null) => Array<Record<string, unknown>>;
|
||||
extractExecutionState: (normalizedPayload: NormalizeResponsePayload["normalized"]) => unknown;
|
||||
buildDebugRoutes: (routeSummary: RouteHintSummary | null) => AssistantDebugRouteRecord[];
|
||||
extractExecutionState: (normalizedPayload: NormalizeResponsePayload["normalized"]) => AssistantExecutionStateRecord[];
|
||||
sanitizeReply: (value: string, fallback?: string) => string;
|
||||
persistInvestigationState: (sessionId: string, snapshot: InvestigationStateWithProblemUnits) => void;
|
||||
nowIso?: () => string;
|
||||
|
|
@ -99,13 +104,13 @@ export interface AssistantDeepTurnPackagingRuntimeOutput {
|
|||
investigationStateSnapshot: InvestigationStateWithProblemUnits | null;
|
||||
droppedIntentSegments: string[];
|
||||
analysisContextForContract: AssistantAnalysisContextForContract | null;
|
||||
routesForDebug: Array<Record<string, unknown>>;
|
||||
resolvedExecutionState: unknown;
|
||||
routesForDebug: AssistantDebugRouteRecord[];
|
||||
resolvedExecutionState: AssistantExecutionStateRecord[];
|
||||
safeAssistantReplyBase: string;
|
||||
safeAssistantReply: string;
|
||||
debug: AssistantDebugPayload;
|
||||
assistantItem: AssistantConversationItem;
|
||||
deepAnalysisLogDetails: Record<string, unknown>;
|
||||
deepAnalysisLogDetails: DeepAnalysisLogDetails;
|
||||
}
|
||||
|
||||
export function runAssistantDeepTurnPackagingRuntime(
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
import type { AssistantRequirement } from "../types/assistant";
|
||||
import type { NormalizeResponsePayload, RouteHintSummary } from "../types/normalizer";
|
||||
import type { AssistantExecutionPlanItem } from "./assistantQueryPlanning";
|
||||
import type { DomainPolarityGuardAudit, TemporalGuardAudit } from "./assistantRuntimeGuards";
|
||||
|
||||
export interface AssistantRequirementExtractionLike {
|
||||
requirements: AssistantRequirement[];
|
||||
|
|
@ -9,7 +10,15 @@ export interface AssistantRequirementExtractionLike {
|
|||
|
||||
export interface AssistantPlanEnforcementAuditLike {
|
||||
executionPlan: AssistantExecutionPlanItem[];
|
||||
audit: Record<string, unknown> | null;
|
||||
audit: AssistantLiveRoutePlanAudit | null;
|
||||
}
|
||||
|
||||
export interface AssistantLiveRoutePlanAudit extends Record<string, unknown> {
|
||||
required_live_calls: string[];
|
||||
route_adjustments_applied: number;
|
||||
rescued_no_route_fragments: number;
|
||||
replaced_routes: string[];
|
||||
route_gap_reason: string | null;
|
||||
}
|
||||
|
||||
export interface BuildAssistantDeepTurnExecutionPlanInput {
|
||||
|
|
@ -17,8 +26,8 @@ export interface BuildAssistantDeepTurnExecutionPlanInput {
|
|||
normalizedPayload: NormalizeResponsePayload["normalized"];
|
||||
userMessage: string;
|
||||
claimType: string;
|
||||
temporalGuard: unknown;
|
||||
domainPolarityGuardInitial: unknown;
|
||||
temporalGuard: TemporalGuardAudit;
|
||||
domainPolarityGuardInitial: DomainPolarityGuardAudit;
|
||||
extractRequirements: (
|
||||
routeSummary: RouteHintSummary | null,
|
||||
normalizedPayload: NormalizeResponsePayload["normalized"],
|
||||
|
|
@ -33,20 +42,20 @@ export interface BuildAssistantDeepTurnExecutionPlanInput {
|
|||
enforceRbpLiveRoutePlan: (input: {
|
||||
executionPlan: AssistantExecutionPlanItem[];
|
||||
claimType: string;
|
||||
temporalGuard: unknown;
|
||||
temporalGuard: TemporalGuardAudit;
|
||||
}) => AssistantPlanEnforcementAuditLike;
|
||||
enforceFaLiveRoutePlan: (input: {
|
||||
executionPlan: AssistantExecutionPlanItem[];
|
||||
claimType: string;
|
||||
temporalGuard: unknown;
|
||||
temporalGuard: TemporalGuardAudit;
|
||||
}) => AssistantPlanEnforcementAuditLike;
|
||||
applyTemporalHintToExecutionPlan: (
|
||||
executionPlan: AssistantExecutionPlanItem[],
|
||||
temporalGuard: unknown
|
||||
temporalGuard: TemporalGuardAudit
|
||||
) => AssistantExecutionPlanItem[];
|
||||
applyPolarityHintToExecutionPlan: (
|
||||
executionPlan: AssistantExecutionPlanItem[],
|
||||
domainPolarityGuardInitial: unknown
|
||||
domainPolarityGuardInitial: DomainPolarityGuardAudit
|
||||
) => AssistantExecutionPlanItem[];
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
import type { NormalizeResponsePayload, RouteHintSummary } from "../types/normalizer";
|
||||
import type { AssistantDebugRouteRecord, AssistantExecutionStateRecord } from "../types/assistant";
|
||||
|
||||
export interface AssistantRuntimeAnalysisContextForPrePackaging {
|
||||
active: boolean;
|
||||
|
|
@ -23,16 +24,16 @@ export interface BuildAssistantDeepTurnPrePackagingContextInput {
|
|||
runtimeAnalysisContext: AssistantRuntimeAnalysisContextForPrePackaging;
|
||||
assistantReply: string;
|
||||
extractDroppedIntentSegments: (normalizedPayload: NormalizeResponsePayload["normalized"]) => string[];
|
||||
buildDebugRoutes: (routeSummary: RouteHintSummary | null) => Array<Record<string, unknown>>;
|
||||
extractExecutionState: (normalizedPayload: NormalizeResponsePayload["normalized"]) => unknown;
|
||||
buildDebugRoutes: (routeSummary: RouteHintSummary | null) => AssistantDebugRouteRecord[];
|
||||
extractExecutionState: (normalizedPayload: NormalizeResponsePayload["normalized"]) => AssistantExecutionStateRecord[];
|
||||
sanitizeReply: (value: string, fallback?: string) => string;
|
||||
}
|
||||
|
||||
export interface AssistantDeepTurnPrePackagingContext {
|
||||
droppedIntentSegments: string[];
|
||||
analysisContextForContract: AssistantAnalysisContextForContract | null;
|
||||
routesForDebug: Array<Record<string, unknown>>;
|
||||
resolvedExecutionState: unknown;
|
||||
routesForDebug: AssistantDebugRouteRecord[];
|
||||
resolvedExecutionState: AssistantExecutionStateRecord[];
|
||||
safeAssistantReplyBase: string;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,12 @@
|
|||
import type { AssistantDebugPayload, AssistantMessageResponsePayload } from "../types/assistant";
|
||||
import type {
|
||||
AssistantAddressRuntimeMetaForDeep,
|
||||
AssistantDebugPayload,
|
||||
AssistantDebugRouteRecord,
|
||||
AssistantExecutionStateRecord,
|
||||
AssistantMessageResponsePayload,
|
||||
FaLiveRouteAuditDebug,
|
||||
RbpLiveRouteAuditDebug
|
||||
} from "../types/assistant";
|
||||
import type { NormalizeResponsePayload, RouteHintSummary } from "../types/normalizer";
|
||||
import type { InvestigationStateWithProblemUnits } from "../types/stage2ProblemUnits";
|
||||
import {
|
||||
|
|
@ -7,10 +15,23 @@ import {
|
|||
type AssistantDeepTurnPackagingRuntimeOutput
|
||||
} from "./assistantDeepTurnPackagingRuntimeAdapter";
|
||||
import type { RunAssistantDeepTurnAnalysisRuntimeOutput } from "./assistantDeepTurnAnalysisRuntimeAdapter";
|
||||
import type { AssistantExecutionPlanItem } from "./assistantQueryPlanning";
|
||||
import {
|
||||
finalizeAssistantDeepTurn,
|
||||
type FinalizeAssistantDeepTurnInput
|
||||
} from "./assistantDeepTurnFinalizeRuntimeAdapter";
|
||||
import type { ClaimBoundAnchorAudit, TargetedEvidenceAcquisitionAudit } from "./assistantClaimBoundEvidence";
|
||||
import type { CompanyAnchorSet } from "./companyAnchorResolver";
|
||||
import type {
|
||||
DomainPolarityGuardAudit,
|
||||
EvidenceAdmissibilityAudit,
|
||||
GroundedAnswerEligibilityAudit,
|
||||
TemporalGuardAudit
|
||||
} from "./assistantRuntimeGuards";
|
||||
import type {
|
||||
AssistantRetrievalCallRecord,
|
||||
AssistantRetrievalRawResultRecord
|
||||
} from "./assistantDeepTurnRetrievalRuntimeAdapter";
|
||||
|
||||
export interface RunAssistantDeepTurnResponseRuntimeInput {
|
||||
featureInvestigationStateV1: boolean;
|
||||
|
|
@ -27,16 +48,16 @@ export interface RunAssistantDeepTurnResponseRuntimeInput {
|
|||
};
|
||||
normalizedQuestion: string;
|
||||
routeSummary: RouteHintSummary | null;
|
||||
executionPlan: unknown[];
|
||||
executionPlan: AssistantExecutionPlanItem[];
|
||||
requirementExtractionRequirements: AssistantDeepTurnPackagingRuntimeInput["requirementExtractionRequirements"];
|
||||
coverageEvaluationRequirements: AssistantDeepTurnPackagingRuntimeInput["coverageEvaluationRequirements"];
|
||||
coverageReport: AssistantDeepTurnPackagingRuntimeInput["coverageReport"];
|
||||
groundingCheck: AssistantDeepTurnPackagingRuntimeInput["groundingCheck"];
|
||||
retrievalCalls: unknown[];
|
||||
retrievalResultsRaw: unknown[];
|
||||
retrievalCalls: AssistantRetrievalCallRecord[];
|
||||
retrievalResultsRaw: AssistantRetrievalRawResultRecord[];
|
||||
retrievalResults: AssistantDeepTurnPackagingRuntimeInput["retrievalResults"];
|
||||
questionTypeClass: AssistantDeepTurnPackagingRuntimeInput["questionTypeClass"];
|
||||
companyAnchors: RunAssistantDeepTurnAnalysisRuntimeOutput["companyAnchors"];
|
||||
companyAnchors: CompanyAnchorSet | null;
|
||||
runtimeAnalysisContext: {
|
||||
active: boolean;
|
||||
as_of_date: string | null;
|
||||
|
|
@ -46,22 +67,22 @@ export interface RunAssistantDeepTurnResponseRuntimeInput {
|
|||
snapshot_mode?: "auto" | "force_snapshot" | "force_live";
|
||||
};
|
||||
businessScopeResolution: RunAssistantDeepTurnAnalysisRuntimeOutput["businessScopeResolution"];
|
||||
temporalGuard: unknown;
|
||||
polarityAudit: unknown;
|
||||
claimAnchorAudit: unknown;
|
||||
targetedEvidenceAudit: unknown;
|
||||
evidenceAdmissibilityGateAudit: unknown;
|
||||
rbpLiveRouteAudit: unknown;
|
||||
faLiveRouteAudit: unknown;
|
||||
groundedAnswerEligibilityGuard: unknown;
|
||||
temporalGuard: TemporalGuardAudit;
|
||||
polarityAudit: DomainPolarityGuardAudit;
|
||||
claimAnchorAudit: ClaimBoundAnchorAudit;
|
||||
targetedEvidenceAudit: TargetedEvidenceAcquisitionAudit;
|
||||
evidenceAdmissibilityGateAudit: EvidenceAdmissibilityAudit;
|
||||
rbpLiveRouteAudit: RbpLiveRouteAuditDebug | null;
|
||||
faLiveRouteAudit: FaLiveRouteAuditDebug | null;
|
||||
groundedAnswerEligibilityGuard: GroundedAnswerEligibilityAudit;
|
||||
followupStateUsage: AssistantDeepTurnPackagingRuntimeInput["followupStateUsage"];
|
||||
followupApplied: boolean;
|
||||
composition: AssistantDeepTurnPackagingRuntimeInput["composition"];
|
||||
previousInvestigationState: AssistantDeepTurnPackagingRuntimeInput["previousInvestigationState"];
|
||||
addressRuntimeMetaForDeep: AssistantDeepTurnPackagingRuntimeInput["addressRuntimeMetaForDeep"];
|
||||
extractDroppedIntentSegments: (normalizedPayload: NormalizeResponsePayload["normalized"]) => string[];
|
||||
buildDebugRoutes: (routeSummary: RouteHintSummary | null) => Array<Record<string, unknown>>;
|
||||
extractExecutionState: (normalizedPayload: NormalizeResponsePayload["normalized"]) => unknown[];
|
||||
buildDebugRoutes: (routeSummary: RouteHintSummary | null) => AssistantDebugRouteRecord[];
|
||||
extractExecutionState: (normalizedPayload: NormalizeResponsePayload["normalized"]) => AssistantExecutionStateRecord[];
|
||||
sanitizeReply: (value: string, fallback?: string) => string;
|
||||
persistInvestigationState: (sessionId: string, snapshot: InvestigationStateWithProblemUnits) => void;
|
||||
messageIdFactory: () => string;
|
||||
|
|
@ -81,14 +102,7 @@ export interface RunAssistantDeepTurnResponseRuntimeOutput {
|
|||
debug: AssistantDebugPayload;
|
||||
}
|
||||
|
||||
function toRecordObject(value: unknown): Record<string, unknown> | null {
|
||||
if (!value || typeof value !== "object") {
|
||||
return null;
|
||||
}
|
||||
return value as Record<string, unknown>;
|
||||
}
|
||||
|
||||
function toNullableString(value: unknown): string | null {
|
||||
function toNullableString(value: string | null | undefined): string | null {
|
||||
if (typeof value !== "string") {
|
||||
return null;
|
||||
}
|
||||
|
|
@ -96,7 +110,7 @@ function toNullableString(value: unknown): string | null {
|
|||
return trimmed.length > 0 ? trimmed : null;
|
||||
}
|
||||
|
||||
function toStringArray(value: unknown): string[] {
|
||||
function toStringArray(value: string[] | null | undefined): string[] {
|
||||
if (!Array.isArray(value)) {
|
||||
return [];
|
||||
}
|
||||
|
|
@ -105,44 +119,30 @@ function toStringArray(value: unknown): string[] {
|
|||
.filter((item) => item.length > 0);
|
||||
}
|
||||
|
||||
function toSnapshotMode(value: unknown): "auto" | "force_snapshot" | "force_live" {
|
||||
function toSnapshotMode(value: string | null | undefined): "auto" | "force_snapshot" | "force_live" {
|
||||
return value === "force_snapshot" || value === "force_live" ? value : "auto";
|
||||
}
|
||||
|
||||
function normalizeExecutionPlan(value: unknown[]): AssistantDeepTurnPackagingRuntimeInput["executionPlan"] {
|
||||
function normalizeExecutionPlan(
|
||||
value: AssistantExecutionPlanItem[]
|
||||
): AssistantDeepTurnPackagingRuntimeInput["executionPlan"] {
|
||||
if (!Array.isArray(value)) {
|
||||
return [];
|
||||
}
|
||||
return value.map((item, index) => {
|
||||
const source = toRecordObject(item);
|
||||
return {
|
||||
fragment_id: toNullableString(source?.fragment_id) ?? `fragment_${index + 1}`,
|
||||
requirement_ids: toStringArray(source?.requirement_ids),
|
||||
route: toNullableString(source?.route) ?? "unknown_route",
|
||||
should_execute: Boolean(source?.should_execute),
|
||||
no_route_reason: toNullableString(source?.no_route_reason),
|
||||
clarification_reason: toNullableString(source?.clarification_reason)
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
function normalizeRecordArray(value: unknown[]): Array<Record<string, unknown>> {
|
||||
if (!Array.isArray(value)) {
|
||||
return [];
|
||||
}
|
||||
return value
|
||||
.map((item) => toRecordObject(item))
|
||||
.filter((item): item is Record<string, unknown> => Boolean(item));
|
||||
}
|
||||
|
||||
function normalizeRecord(value: unknown): Record<string, unknown> {
|
||||
return toRecordObject(value) ?? {};
|
||||
return value.map((item, index) => ({
|
||||
fragment_id: toNullableString(item.fragment_id) ?? `fragment_${index + 1}`,
|
||||
requirement_ids: toStringArray(item.requirement_ids),
|
||||
route: toNullableString(item.route) ?? "unknown_route",
|
||||
should_execute: Boolean(item.should_execute),
|
||||
no_route_reason: toNullableString(item.no_route_reason ?? null),
|
||||
clarification_reason: toNullableString(item.clarification_reason ?? null)
|
||||
}));
|
||||
}
|
||||
|
||||
function normalizeRuntimeAnalysisContext(
|
||||
value: unknown
|
||||
value: RunAssistantDeepTurnResponseRuntimeInput["runtimeAnalysisContext"] | null | undefined
|
||||
): AssistantDeepTurnPackagingRuntimeInput["runtimeAnalysisContext"] {
|
||||
const source = toRecordObject(value);
|
||||
const source = value;
|
||||
return {
|
||||
active: Boolean(source?.active),
|
||||
as_of_date: toNullableString(source?.as_of_date),
|
||||
|
|
@ -154,9 +154,9 @@ function normalizeRuntimeAnalysisContext(
|
|||
}
|
||||
|
||||
function normalizeBusinessScopeResolution(
|
||||
value: unknown
|
||||
value: RunAssistantDeepTurnAnalysisRuntimeOutput["businessScopeResolution"] | null | undefined
|
||||
): AssistantDeepTurnPackagingRuntimeInput["businessScopeResolution"] {
|
||||
const source = toRecordObject(value);
|
||||
const source = value;
|
||||
return {
|
||||
business_scope_raw: toStringArray(source?.business_scope_raw),
|
||||
business_scope_resolved: toStringArray(source?.business_scope_resolved),
|
||||
|
|
@ -166,9 +166,9 @@ function normalizeBusinessScopeResolution(
|
|||
}
|
||||
|
||||
function normalizeAddressRuntimeMetaForDeep(
|
||||
value: unknown
|
||||
): AssistantDeepTurnPackagingRuntimeInput["addressRuntimeMetaForDeep"] {
|
||||
const source = toRecordObject(value);
|
||||
value: AssistantAddressRuntimeMetaForDeep | null | undefined
|
||||
): AssistantAddressRuntimeMetaForDeep | null {
|
||||
const source = value;
|
||||
if (!source) {
|
||||
return null;
|
||||
}
|
||||
|
|
@ -180,8 +180,8 @@ function normalizeAddressRuntimeMetaForDeep(
|
|||
fallbackRuleHit: toNullableString(source.fallbackRuleHit),
|
||||
toolGateDecision: toNullableString(source.toolGateDecision),
|
||||
toolGateReason: toNullableString(source.toolGateReason),
|
||||
predecomposeContract: toRecordObject(source.predecomposeContract),
|
||||
orchestrationContract: toRecordObject(source.orchestrationContract)
|
||||
predecomposeContract: source.predecomposeContract ?? null,
|
||||
orchestrationContract: source.orchestrationContract ?? null
|
||||
};
|
||||
}
|
||||
|
||||
|
|
@ -204,21 +204,21 @@ export function runAssistantDeepTurnResponseRuntime(
|
|||
coverageEvaluationRequirements: input.coverageEvaluationRequirements,
|
||||
coverageReport: input.coverageReport,
|
||||
groundingCheck: input.groundingCheck,
|
||||
retrievalCalls: normalizeRecordArray(input.retrievalCalls),
|
||||
retrievalResultsRaw: Array.isArray(input.retrievalResultsRaw) ? input.retrievalResultsRaw : [],
|
||||
retrievalCalls: input.retrievalCalls,
|
||||
retrievalResultsRaw: input.retrievalResultsRaw,
|
||||
retrievalResults: input.retrievalResults,
|
||||
questionTypeClass: input.questionTypeClass,
|
||||
companyAnchors: input.companyAnchors,
|
||||
runtimeAnalysisContext: normalizeRuntimeAnalysisContext(input.runtimeAnalysisContext),
|
||||
businessScopeResolution: normalizeBusinessScopeResolution(input.businessScopeResolution),
|
||||
temporalGuard: normalizeRecord(input.temporalGuard),
|
||||
polarityAudit: normalizeRecord(input.polarityAudit),
|
||||
claimAnchorAudit: normalizeRecord(input.claimAnchorAudit),
|
||||
temporalGuard: input.temporalGuard,
|
||||
polarityAudit: input.polarityAudit,
|
||||
claimAnchorAudit: input.claimAnchorAudit,
|
||||
targetedEvidenceAudit: input.targetedEvidenceAudit,
|
||||
evidenceAdmissibilityGateAudit: input.evidenceAdmissibilityGateAudit,
|
||||
rbpLiveRouteAudit: input.rbpLiveRouteAudit ?? null,
|
||||
faLiveRouteAudit: input.faLiveRouteAudit ?? null,
|
||||
groundedAnswerEligibilityGuard: normalizeRecord(input.groundedAnswerEligibilityGuard),
|
||||
groundedAnswerEligibilityGuard: input.groundedAnswerEligibilityGuard,
|
||||
followupStateUsage: input.followupStateUsage,
|
||||
followupApplied: input.followupApplied,
|
||||
composition: input.composition,
|
||||
|
|
|
|||
|
|
@ -18,10 +18,43 @@ export interface AssistantRetrievalCallRecord {
|
|||
reason: string | null;
|
||||
}
|
||||
|
||||
export type AssistantRetrievalScalar = string | number | boolean | null;
|
||||
export type AssistantRetrievalFieldValue =
|
||||
| AssistantRetrievalScalar
|
||||
| AssistantRetrievalFieldValue[]
|
||||
| { [key: string]: AssistantRetrievalFieldValue };
|
||||
export type AssistantRetrievalRecord = Record<string, AssistantRetrievalFieldValue>;
|
||||
|
||||
export interface AssistantRetrievalRawResultLike {
|
||||
status: "ok" | "empty" | "partial" | "error";
|
||||
result_type: "list" | "summary" | "object" | "chain" | "ranking";
|
||||
items: AssistantRetrievalRecord[];
|
||||
summary: AssistantRetrievalRecord;
|
||||
evidence: AssistantRetrievalRecord[];
|
||||
why_included: string[];
|
||||
selection_reason: string[];
|
||||
risk_factors: string[];
|
||||
business_interpretation: string[];
|
||||
confidence: "high" | "medium" | "low";
|
||||
limitations: string[];
|
||||
errors: string[];
|
||||
}
|
||||
|
||||
export type AssistantRetrievalRawListItem = AssistantRetrievalFieldValue;
|
||||
export type AssistantRetrievalRawList = AssistantRetrievalFieldValue[];
|
||||
|
||||
export type AssistantRetrievalRawResult =
|
||||
| AssistantRetrievalRawResultLike
|
||||
| AssistantRetrievalRawList
|
||||
| string
|
||||
| number
|
||||
| boolean
|
||||
| null;
|
||||
|
||||
export interface AssistantRetrievalRawResultRecord {
|
||||
fragment_id: string;
|
||||
route: string;
|
||||
raw_result: unknown;
|
||||
raw_result: AssistantRetrievalRawResult;
|
||||
}
|
||||
|
||||
export interface AssistantDeepTurnRetrievalExecutionInput {
|
||||
|
|
@ -33,7 +66,7 @@ export interface AssistantDeepTurnRetrievalExecutionInput {
|
|||
options: {
|
||||
temporalHint: AssistantLiveTemporalHint | null;
|
||||
}
|
||||
) => Promise<unknown>;
|
||||
) => Promise<AssistantRetrievalRawResult>;
|
||||
mapNoRouteReason: (reason: string | null) => string;
|
||||
buildSkippedResult: (item: AssistantExecutionPlanItem) => UnifiedRetrievalResult;
|
||||
normalizeRetrievalResultFn?: typeof normalizeRetrievalResult;
|
||||
|
|
@ -45,7 +78,7 @@ export interface AssistantDeepTurnRetrievalExecutionOutput {
|
|||
retrievalResults: UnifiedRetrievalResult[];
|
||||
}
|
||||
|
||||
function buildRouteExecutorErrorRawResult(route: string, message: string): Record<string, unknown> {
|
||||
function buildRouteExecutorErrorRawResult(route: string, message: string): AssistantRetrievalRawResult {
|
||||
return {
|
||||
status: "error",
|
||||
result_type: "summary",
|
||||
|
|
@ -64,6 +97,24 @@ function buildRouteExecutorErrorRawResult(route: string, message: string): Recor
|
|||
};
|
||||
}
|
||||
|
||||
function normalizeRawResult(
|
||||
value: AssistantRetrievalRawResult | object | null | undefined
|
||||
): AssistantRetrievalRawResult {
|
||||
if (value === null || value === undefined) {
|
||||
return null;
|
||||
}
|
||||
if (typeof value === "string" || typeof value === "number" || typeof value === "boolean") {
|
||||
return value;
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
return value as AssistantRetrievalRawList;
|
||||
}
|
||||
if (typeof value === "object") {
|
||||
return value as AssistantRetrievalRawResultLike;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export async function executeAssistantDeepTurnRetrievalPlan(
|
||||
input: AssistantDeepTurnRetrievalExecutionInput
|
||||
): Promise<AssistantDeepTurnRetrievalExecutionOutput> {
|
||||
|
|
@ -99,13 +150,14 @@ export async function executeAssistantDeepTurnRetrievalPlan(
|
|||
const raw = await input.executeRouteRuntime(planItem.route, planItem.fragment_text, {
|
||||
temporalHint: input.liveTemporalHint
|
||||
});
|
||||
const normalizedRaw = normalizeRawResult(raw);
|
||||
retrievalResultsRaw.push({
|
||||
fragment_id: planItem.fragment_id,
|
||||
route: planItem.route,
|
||||
raw_result: raw
|
||||
raw_result: normalizedRaw
|
||||
});
|
||||
retrievalResults.push(
|
||||
normalizeRetrievalResultSafe(planItem.fragment_id, planItem.requirement_ids, planItem.route, raw)
|
||||
normalizeRetrievalResultSafe(planItem.fragment_id, planItem.requirement_ids, planItem.route, normalizedRaw)
|
||||
);
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ import {
|
|||
buildAssistantEvidenceBundleContractV1,
|
||||
type AssistantEvidenceBundleContractV1
|
||||
} from "./assistantOrchestrationContracts";
|
||||
import type { AssistantRetrievalCallRecord } from "./assistantDeepTurnRetrievalRuntimeAdapter";
|
||||
|
||||
type RetrievalStatusItem = AssistantDebugPayload["retrieval_status"][number];
|
||||
|
||||
|
|
@ -22,7 +23,7 @@ function buildRetrievalStatus(retrievalResults: UnifiedRetrievalResult[]): Retri
|
|||
}
|
||||
|
||||
export function assembleAssistantEvidenceBundle(input: {
|
||||
retrievalCalls: Array<Record<string, unknown>>;
|
||||
retrievalCalls: AssistantRetrievalCallRecord[];
|
||||
retrievalResults: UnifiedRetrievalResult[];
|
||||
}): AssistantEvidenceBundleAssembly {
|
||||
const retrievalResults = Array.isArray(input.retrievalResults) ? input.retrievalResults : [];
|
||||
|
|
|
|||
|
|
@ -1,4 +1,33 @@
|
|||
import type { AnswerGroundingCheck, RequirementCoverageReport } from "../types/assistant";
|
||||
import type {
|
||||
AssistantDebugRouteRecord,
|
||||
AssistantExecutionStateRecord,
|
||||
AssistantFallbackType,
|
||||
AssistantProblemAnswerMode,
|
||||
AssistantReplyType,
|
||||
AssistantRequirement,
|
||||
UnifiedRetrievalResult,
|
||||
AnswerGroundingCheck,
|
||||
FaLiveRouteAuditDebug,
|
||||
RbpLiveRouteAuditDebug,
|
||||
RequirementCoverageReport
|
||||
} from "../types/assistant";
|
||||
import type { AnswerStructureV11 } from "../types/stage1Contracts";
|
||||
import type { InvestigationStateWithProblemUnits } from "../types/stage2ProblemUnits";
|
||||
import type { ClaimBoundAnchorAudit, TargetedEvidenceAcquisitionAudit } from "./assistantClaimBoundEvidence";
|
||||
import type { AssistantContractsBundleV1 } from "./assistantContractsBundleAssembler";
|
||||
import type { AssistantOutcomeClassV1 } from "./assistantOrchestrationContracts";
|
||||
import type { AssistantFollowupUsage } from "./assistantFollowupUsage";
|
||||
import type { CompanyAnchorSet } from "./companyAnchorResolver";
|
||||
import type {
|
||||
AssistantRetrievalCallRecord,
|
||||
AssistantRetrievalRawResultRecord
|
||||
} from "./assistantDeepTurnRetrievalRuntimeAdapter";
|
||||
import type {
|
||||
DomainPolarityGuardAudit,
|
||||
EvidenceAdmissibilityAudit,
|
||||
GroundedAnswerEligibilityAudit,
|
||||
TemporalGuardAudit
|
||||
} from "./assistantRuntimeGuards";
|
||||
|
||||
export interface DeepAnalysisMessageLogDetailsInput {
|
||||
sessionId: string;
|
||||
|
|
@ -6,18 +35,18 @@ export interface DeepAnalysisMessageLogDetailsInput {
|
|||
userMessage: string;
|
||||
normalizerOutput: unknown;
|
||||
executionPlan: Array<Record<string, unknown>>;
|
||||
resolvedExecutionState: unknown;
|
||||
routes: Array<Record<string, unknown>>;
|
||||
retrievalCalls: Array<Record<string, unknown>>;
|
||||
retrievalResultsRaw: unknown[];
|
||||
retrievalResultsNormalized: unknown[];
|
||||
requirementsExtracted: unknown[];
|
||||
resolvedExecutionState: AssistantExecutionStateRecord[];
|
||||
routes: AssistantDebugRouteRecord[];
|
||||
retrievalCalls: AssistantRetrievalCallRecord[];
|
||||
retrievalResultsRaw: AssistantRetrievalRawResultRecord[];
|
||||
retrievalResultsNormalized: UnifiedRetrievalResult[];
|
||||
requirementsExtracted: AssistantRequirement[];
|
||||
coverageReport: RequirementCoverageReport;
|
||||
groundingCheck: AnswerGroundingCheck;
|
||||
replyType: string;
|
||||
replyType: AssistantReplyType;
|
||||
droppedIntentSegments: string[];
|
||||
questionTypeClass: string;
|
||||
companyAnchors: unknown;
|
||||
companyAnchors: CompanyAnchorSet | null;
|
||||
runtimeAnalysisContext: {
|
||||
active: boolean;
|
||||
as_of_date: string | null;
|
||||
|
|
@ -32,26 +61,26 @@ export interface DeepAnalysisMessageLogDetailsInput {
|
|||
company_grounding_applied?: boolean;
|
||||
scope_resolution_reason?: string[];
|
||||
};
|
||||
temporalGuard: Record<string, unknown>;
|
||||
polarityAudit: Record<string, unknown>;
|
||||
claimAnchorAudit: Record<string, unknown>;
|
||||
targetedEvidenceAudit: unknown;
|
||||
evidenceAdmissibilityGateAudit: unknown;
|
||||
rbpLiveRouteAudit: unknown | null;
|
||||
faLiveRouteAudit: unknown | null;
|
||||
groundedAnswerEligibilityGuard: Record<string, unknown>;
|
||||
followupStateUsage: unknown | null;
|
||||
temporalGuard: TemporalGuardAudit;
|
||||
polarityAudit: DomainPolarityGuardAudit;
|
||||
claimAnchorAudit: ClaimBoundAnchorAudit;
|
||||
targetedEvidenceAudit: TargetedEvidenceAcquisitionAudit;
|
||||
evidenceAdmissibilityGateAudit: EvidenceAdmissibilityAudit;
|
||||
rbpLiveRouteAudit: RbpLiveRouteAuditDebug | null;
|
||||
faLiveRouteAudit: FaLiveRouteAuditDebug | null;
|
||||
groundedAnswerEligibilityGuard: GroundedAnswerEligibilityAudit;
|
||||
followupStateUsage: AssistantFollowupUsage | null;
|
||||
compositionDebug: {
|
||||
problem_centric_answer_applied?: boolean;
|
||||
problem_units_used_count?: number;
|
||||
problem_answer_mode?: string;
|
||||
problem_answer_mode?: AssistantProblemAnswerMode;
|
||||
problem_unit_ids_used?: string[];
|
||||
fallback_type?: string;
|
||||
fallback_type?: AssistantFallbackType;
|
||||
};
|
||||
outcomeClassV1: unknown;
|
||||
assistantOrchestrationContractsV1: unknown;
|
||||
answerStructureV11: unknown;
|
||||
investigationStateSnapshot: unknown;
|
||||
outcomeClassV1: AssistantOutcomeClassV1;
|
||||
assistantOrchestrationContractsV1: AssistantContractsBundleV1["assistantOrchestrationContractsV1"];
|
||||
answerStructureV11: AnswerStructureV11 | null;
|
||||
investigationStateSnapshot: InvestigationStateWithProblemUnits | null;
|
||||
assistantReply: string;
|
||||
traceId: string;
|
||||
}
|
||||
|
|
@ -77,7 +106,9 @@ function resolveCoverageStatus(coverageReport: RequirementCoverageReport): "full
|
|||
: "partial_or_limited";
|
||||
}
|
||||
|
||||
export function buildDeepAnalysisProcessedLogDetails(input: DeepAnalysisMessageLogDetailsInput): Record<string, unknown> {
|
||||
export type DeepAnalysisLogDetails = Record<string, unknown>;
|
||||
|
||||
export function buildDeepAnalysisProcessedLogDetails(input: DeepAnalysisMessageLogDetailsInput): DeepAnalysisLogDetails {
|
||||
const analysisContext = toAnalysisContext(input.runtimeAnalysisContext);
|
||||
return {
|
||||
session_id: input.sessionId,
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ import type {
|
|||
UnifiedRetrievalResult
|
||||
} from "../types/assistant";
|
||||
import type { NormalizedPayload, RouteHintSummary } from "../types/normalizer";
|
||||
import type { AssistantRetrievalCallRecord } from "./assistantDeepTurnRetrievalRuntimeAdapter";
|
||||
|
||||
export type AssistantOutcomeClassV1 =
|
||||
| "FULLY_ANSWERED"
|
||||
|
|
@ -241,7 +242,7 @@ export function buildAssistantExecutionPlanContractV1(input: {
|
|||
}
|
||||
|
||||
export function buildAssistantEvidenceBundleContractV1(input: {
|
||||
retrievalCalls: Array<Record<string, unknown>>;
|
||||
retrievalCalls: AssistantRetrievalCallRecord[];
|
||||
retrievalResults: UnifiedRetrievalResult[];
|
||||
}): AssistantEvidenceBundleContractV1 {
|
||||
const retrievalResults = Array.isArray(input.retrievalResults) ? input.retrievalResults : [];
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import type { AssistantDebugRouteRecord } from "../types/assistant";
|
||||
import type { RouteHintSummary } from "../types/normalizer";
|
||||
|
||||
interface FragmentLike {
|
||||
|
|
@ -106,7 +107,7 @@ export function buildExecutionPlanFromRoute(input: {
|
|||
export function buildDebugRoutesFromRoute(input: {
|
||||
routeSummary: RouteHintSummary | null;
|
||||
resolveLegacyRouteReason: (route: string) => string;
|
||||
}): Array<Record<string, unknown>> {
|
||||
}): AssistantDebugRouteRecord[] {
|
||||
if (!input.routeSummary) {
|
||||
return [];
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ import type { CompanyAnchorSet } from "./companyAnchorResolver";
|
|||
import type { EvidenceItem } from "../types/stage1Contracts";
|
||||
import type { ProblemUnit } from "../types/stage2ProblemUnits";
|
||||
import type { ClaimBoundAnchorAudit } from "./assistantClaimBoundEvidence";
|
||||
import iconv from "iconv-lite";
|
||||
|
||||
type P0DomainHint =
|
||||
| "settlements_60_62"
|
||||
|
|
@ -895,12 +896,70 @@ export interface DomainPolarityGuardAudit {
|
|||
reason_codes: string[];
|
||||
}
|
||||
|
||||
function mojibakeScoreForRuntimeGuards(value: string): number {
|
||||
const source = String(value ?? "");
|
||||
const cyrillic = (source.match(/[А-Яа-яЁё]/g) ?? []).length;
|
||||
const latin = (source.match(/[A-Za-z]/g) ?? []).length;
|
||||
const hardMarkers = (source.match(/[ѓ“‚„…†‡€‰‹‰ЉЊ‹Џ‘’“”•–—™љ›њћџ]/g) ?? []).length;
|
||||
const pairMarkers = (source.match(/(?:Р.|С.|Гђ.|Г‘.)/g) ?? []).length;
|
||||
const doubleEncodedMarkers = (source.match(/(?:Р“.|Р’.|Гѓ.|Г‚.)/gu) ?? []).length;
|
||||
return cyrillic + latin - hardMarkers * 3 - pairMarkers * 2 - doubleEncodedMarkers * 2;
|
||||
}
|
||||
|
||||
function looksLikeMojibakeForRuntimeGuards(value: string): boolean {
|
||||
const source = String(value ?? "");
|
||||
if (!source.trim()) {
|
||||
return false;
|
||||
}
|
||||
if (/[ѓ“‚„…†‡€‰‹‰ЉЊ‹Џ‘’“”•–—™љ›њћџ]/.test(source)) {
|
||||
return true;
|
||||
}
|
||||
if ((source.match(/(?:Р.|С.|Гђ.|Г‘.)/g) ?? []).length >= 2) {
|
||||
return true;
|
||||
}
|
||||
return (source.match(/(?:Р“.|Р’.|Гѓ.|Г‚.)/gu) ?? []).length >= 2;
|
||||
}
|
||||
|
||||
function repairRuntimeGuardsMojibake(value: string): string {
|
||||
const source = String(value ?? "");
|
||||
if (!looksLikeMojibakeForRuntimeGuards(source)) {
|
||||
return source;
|
||||
}
|
||||
let candidate = source;
|
||||
for (let pass = 0; pass < 3; pass += 1) {
|
||||
let improved = false;
|
||||
try {
|
||||
const fromWin1251 = iconv.encode(candidate, "win1251").toString("utf8");
|
||||
if (mojibakeScoreForRuntimeGuards(fromWin1251) > mojibakeScoreForRuntimeGuards(candidate)) {
|
||||
candidate = fromWin1251;
|
||||
improved = true;
|
||||
}
|
||||
} catch (_error) {
|
||||
// noop
|
||||
}
|
||||
try {
|
||||
const fromLatin1 = Buffer.from(candidate, "latin1").toString("utf8");
|
||||
if (mojibakeScoreForRuntimeGuards(fromLatin1) > mojibakeScoreForRuntimeGuards(candidate)) {
|
||||
candidate = fromLatin1;
|
||||
improved = true;
|
||||
}
|
||||
} catch (_error) {
|
||||
// noop
|
||||
}
|
||||
if (!improved) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
return candidate;
|
||||
}
|
||||
|
||||
export function resolveDomainPolarityGuard(input: {
|
||||
userMessage: string;
|
||||
companyAnchors?: CompanyAnchorSet | null;
|
||||
focusDomainHint?: string | null;
|
||||
}): DomainPolarityGuardAudit {
|
||||
const lower = String(input.userMessage ?? "").toLowerCase();
|
||||
const repairedMessage = repairRuntimeGuardsMojibake(String(input.userMessage ?? ""));
|
||||
const lower = repairedMessage.toLowerCase();
|
||||
const accountExtraction = extractAccountsFromTextDetailed(lower);
|
||||
const accounts = uniqueStrings([...(input.companyAnchors?.accounts ?? []), ...accountExtraction.resolved_account_anchors]);
|
||||
const prefixes = new Set(accounts.map((item) => accountPrefix(item)).filter((item): item is string => Boolean(item)));
|
||||
|
|
@ -1683,7 +1742,8 @@ export function applyEligibilityToGroundingCheck<T extends { status: string; rea
|
|||
const reasonMap: Record<string, string> = {
|
||||
admissible_evidence_count_zero: "Недостаточно подтвержденных данных для уверенного ответа.",
|
||||
critical_domain_or_account_contradiction: "Есть противоречие по выбранному домену или контуру счета.",
|
||||
temporal_guard_failed_out_of_snapshot_window: "Запрошенный период выходит за доступный срез данных.",
|
||||
temporal_guard_failed_out_of_snapshot_window:
|
||||
"Запрошенный период выходит за доступный срез данных. Temporal anchor outside snapshot window.",
|
||||
temporal_guard_ambiguous_limited: "Период в вопросе определен недостаточно точно.",
|
||||
business_scope_generic_unresolved: "Не удалось надежно привязать вопрос к конкретному бизнес-контексту.",
|
||||
polarity_guard_limited_unresolved_polarity: "Не удалось однозначно определить сторону расчета (нам должны или мы должны).",
|
||||
|
|
|
|||
|
|
@ -1040,7 +1040,12 @@ function hasCrossScopeConflictWithState(userMessage, state) {
|
|||
const inferredDomain = inferP0DomainFromMessage(userMessage);
|
||||
const stateDomain = compactWhitespace(state.followup_context?.active_domain ?? state.focus.domain ?? "");
|
||||
if (inferredDomain && stateDomain && inferredDomain !== stateDomain) {
|
||||
return true;
|
||||
const followupDomainRefinement = hasFollowupMarker(userMessage) ||
|
||||
hasReferentialPointer(userMessage) ||
|
||||
hasPeriodLiteral(userMessage);
|
||||
if (!followupDomainRefinement) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
const explicitAccounts = extractAccountTokens(userMessage);
|
||||
const fallbackAccounts = explicitAccounts.length > 0 ? explicitAccounts : extractFollowupAccountAnchorsLoose(userMessage);
|
||||
|
|
@ -1070,9 +1075,11 @@ function inferP0DomainFromMessage(text) {
|
|||
return null;
|
||||
}
|
||||
function hasStrongFollowupAnchors(userMessage, state) {
|
||||
const normalizedMessage = compactWhitespace(repairAddressMojibake(String(userMessage ?? "")).toLowerCase());
|
||||
const periodRefinementCue = /(?:^(?:\u0430\s+)?\u0435\u0441\u043b\u0438|\u0442\u043e\u043b\u044c\u043a\u043e\s+\u0437\u0430|\u0441\u043c\u043e\u0442\u0440\u0435\u0442\u044c|\u043f\u043e\s+\u043f\u0435\u0440\u0438\u043e\u0434\u0443|\u0437\u0430\s+\u0438\u044e\u043d\u044c|\u0437\u0430\s+\u0438\u044e\u043b\u044c)/iu.test(normalizedMessage);
|
||||
const explicitPeriod = extractNormalizedPeriodLiteral(userMessage);
|
||||
if (explicitPeriod && state.focus.period && explicitPeriod !== state.focus.period) {
|
||||
const periodLooksLikeFollowupRefinement = hasFollowupMarker(userMessage) || hasReferentialPointer(userMessage);
|
||||
const periodLooksLikeFollowupRefinement = hasFollowupMarker(userMessage) || hasReferentialPointer(userMessage) || periodRefinementCue;
|
||||
if (!periodLooksLikeFollowupRefinement) {
|
||||
return true;
|
||||
}
|
||||
|
|
@ -2969,26 +2976,33 @@ function resolveAddressToolGateDecision(addressInputMessage, followupContext, ll
|
|||
reason: dataScopeMetaQuery ? "assistant_data_scope_query_detected" : "assistant_capability_query_detected"
|
||||
};
|
||||
}
|
||||
const directDeepAnalysisSignal = hasDirectDeepAnalysisSignal(rawMessageForGate) ||
|
||||
hasDirectDeepAnalysisSignal(repairedInputMessage);
|
||||
const deepAnalysisPreferenceSignal = directDeepAnalysisSignal ||
|
||||
hasDeepAnalysisPreferenceSignal(rawMessageForGate) ||
|
||||
hasDeepAnalysisPreferenceSignal(repairedInputMessage);
|
||||
const modeDetection = (0, addressQueryClassifier_1.detectAddressQuestionMode)(repairedInputMessage || addressInputMessage);
|
||||
const hasClassifierSignal = modeDetection.mode === "address_query";
|
||||
const llmContractMode = toNonEmptyString(llmPreDecomposeMeta?.predecomposeContract?.mode);
|
||||
const llmContractModeConfidence = toNonEmptyString(llmPreDecomposeMeta?.predecomposeContract?.mode_confidence);
|
||||
const llmContractIntent = toNonEmptyString(llmPreDecomposeMeta?.predecomposeContract?.intent);
|
||||
const llmContractIntentConfidence = toNonEmptyString(llmPreDecomposeMeta?.predecomposeContract?.intent_confidence);
|
||||
const llmCanonicalEntitySignal = /(?:\u0437\u0430\u043a\u0430\u0437\u0447\u0438\u043a|\u043f\u043e\u0441\u0442\u0430\u0432\u0449\u0438\u043a|\u043a\u043e\u043d\u0442\u0440\u0430\u0433\u0435\u043d\u0442|\u043a\u043e\u043c\u043f\u0430\u043d|customer|supplier|counterparty|company|vendor|client|\b[a-z]{2,}\b)/iu.test(compactWhitespace(repairedInputMessage.toLowerCase()));
|
||||
const llmCanonicalAppliedSignal = Boolean(llmPreDecomposeMeta?.applied) && llmContractMode !== "deep_analysis";
|
||||
const hasLlmCanonicalSignal = Boolean(llmPreDecomposeMeta?.llmCanonicalCandidateDetected) &&
|
||||
llmContractMode === "address_query" &&
|
||||
llmContractModeConfidence !== "low" &&
|
||||
llmContractIntent !== null &&
|
||||
llmContractIntent !== "unknown" &&
|
||||
llmContractIntentConfidence !== "low";
|
||||
((llmContractMode === "address_query" && llmContractModeConfidence !== "low") ||
|
||||
(llmCanonicalAppliedSignal &&
|
||||
(hasStrongDataIntentSignal(repairedInputMessage) || llmCanonicalEntitySignal || llmContractMode === "unsupported")));
|
||||
const hasLlmCanonicalDataSignal = Boolean(llmPreDecomposeMeta?.llmCanonicalCandidateDetected) &&
|
||||
Boolean(llmPreDecomposeMeta?.applied) &&
|
||||
llmContractMode === "address_query" &&
|
||||
(llmContractMode === "address_query" || llmContractMode === "unsupported" || llmContractMode === null) &&
|
||||
hasStrongDataIntentSignal(repairedInputMessage);
|
||||
const sameDateAccountFollowupSignal = hasSameDateAccountFollowupSignalForPredecompose(rawMessageForGate) ||
|
||||
hasSameDateAccountFollowupSignalForPredecompose(repairedInputMessage);
|
||||
const hasLexicalAddressSignal = isAddressLlmPreDecomposeCandidate(addressInputMessage) ||
|
||||
isAddressLlmPreDecomposeCandidate(repairedInputMessage) ||
|
||||
hasAccountingSignal(addressInputMessage) ||
|
||||
hasAccountingSignal(repairedInputMessage);
|
||||
hasAccountingSignal(repairedInputMessage) ||
|
||||
sameDateAccountFollowupSignal;
|
||||
const hasUnsupportedLowConfidencePredecomposeSignal = llmContractMode === "unsupported" &&
|
||||
(llmContractModeConfidence === "low" || llmContractModeConfidence === "medium") &&
|
||||
llmContractIntent === "unknown";
|
||||
|
|
@ -3036,6 +3050,89 @@ function resolveAddressToolGateDecision(addressInputMessage, followupContext, ll
|
|||
reason: "no_address_signal_after_l0"
|
||||
};
|
||||
}
|
||||
function hasLooseAllTimeAddressLookupSignal(text) {
|
||||
const repaired = repairAddressMojibake(String(text ?? ""));
|
||||
const normalized = compactWhitespace(repaired.toLowerCase());
|
||||
if (!normalized) {
|
||||
return false;
|
||||
}
|
||||
if (shouldHandleAsAssistantCapabilityMetaQuery(normalized) || hasAssistantDataScopeMetaQuestionSignal(normalized)) {
|
||||
return false;
|
||||
}
|
||||
const hasAllTimeSignal = /(?:\u0437\u0430\s+\u0432\u0435\u0441\u044c\s+\u043f\u0435\u0440\u0438\u043e\u0434|\u0437\u0430\s+\u0432\u0441\u0435\s+\u0432\u0440\u0435\u043c\u044f|\u0437\u0430\s+\u0432\u0441\u044e\s+\u0438\u0441\u0442\u043e\u0440\u0438(?:\u044e|\u0438)|for\s+all\s+time|all\s+time|entire\s+period|full\s+period)/iu.test(normalized);
|
||||
if (!hasAllTimeSignal) {
|
||||
return false;
|
||||
}
|
||||
return /(?:\u0447\u0442\u043e\s+\u0435\u0441\u0442\u044c|\u0447[\u0435\u0451]\s+\u0435\u0441\u0442\u044c|\u043f\u043e\u043a\u0430\u0436\u0438|\u0432\u044b\u0432\u0435\u0434\u0438|\u0434\u0430\u0439|show|list|find)/iu.test(normalized);
|
||||
}
|
||||
function hasDeepSessionContinuationSignal(input) {
|
||||
const sessionItems = Array.isArray(input?.sessionItems) ? input.sessionItems : [];
|
||||
if (sessionItems.length === 0) {
|
||||
return false;
|
||||
}
|
||||
const previousDebug = findLastAssistantLivingChatDebug(sessionItems);
|
||||
if (!previousDebug || typeof previousDebug !== "object") {
|
||||
return false;
|
||||
}
|
||||
const investigationState = previousDebug.investigation_state_snapshot;
|
||||
if (!investigationState || typeof investigationState !== "object") {
|
||||
return false;
|
||||
}
|
||||
const candidateTexts = [
|
||||
input?.rawUserMessage,
|
||||
input?.repairedRawUserMessage,
|
||||
input?.effectiveAddressUserMessage,
|
||||
input?.repairedEffectiveAddressUserMessage
|
||||
]
|
||||
.map((value) => compactWhitespace(repairAddressMojibake(String(value ?? "")).toLowerCase()))
|
||||
.filter((value) => value.length > 0);
|
||||
if (candidateTexts.length === 0) {
|
||||
return false;
|
||||
}
|
||||
return candidateTexts.some((text) => {
|
||||
const hasContinuationCue = /^(?:\u0438|\u0430|\u0442\u0430\u043a\u0436\u0435|\u0435\u0449[\u0435\u0451]|\u0434\u043e\u0431\u0430\u0432\u044c|\u0434\u043e\u043f\u043e\u043b\u043d\u0438|\u0443\u0442\u043e\u0447\u043d\u0438|\u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0438|\u0442\u0435\u043f\u0435\u0440\u044c|then|also|and)\b/iu.test(text) ||
|
||||
/(?:\u043f\u043e\s+\u0442\u043e\u043c\u0443\s+\u0436\u0435|\u043f\u043e\s+\u044d\u0442\u043e\u043c\u0443|\u0432\s+\u044d\u0442\u043e\u043c\s+\u0436\u0435|\u0438\s+\u043f\u043e\s+\u043f\u0435\u0440\u0438\u043e\u0434\u0443|\u0434\u043e\u0431\u0430\u0432\u044c\s+\u0443\u0442\u043e\u0447\u043d\u0435\u043d\u0438\u0435|\u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0438\u043c|\u0430\s+\u0435\u0441\u043b\u0438|\u0435\u0441\u043b\u0438\s+\u0442\u043e\u043b\u044c\u043a\u043e|\u043e\u0442\u0434\u0435\u043b\u044c\u043d\u043e)/iu.test(text);
|
||||
const hasAccountOrPeriodCue = /(?:\u0441\u0447[\u0435\u0451]\u0442|account|\b\d{2}(?:[.,]\d{1,2})?\b|\b20\d{2}(?:[-/.]\d{1,2})?\b|\u043f\u0435\u0440\u0438\u043e\u0434|\u043c\u0435\u0441\u044f\u0446)/iu.test(text);
|
||||
const hasDeepRebindCue = /(?:\u0430\u043c\u043e\u0440\u0442\u0438\u0437|fixed\s*asset|\u043e\u0441\b|\u043d\u0434\u0441|vat|\u0440\u0430\u0437\u0440\u044b\u0432|\u0446\u0435\u043f\u043e\u0447\u043a|\u0430\u043d\u043e\u043c\u0430\u043b|lifecycle|\u043f\u0440\u043e\u0442\u0438\u0432\u043e\u0440\u0435\u0447)/iu.test(text);
|
||||
if (hasContinuationCue && (hasAccountOrPeriodCue || hasDeepRebindCue)) {
|
||||
return true;
|
||||
}
|
||||
return hasDeepRebindCue && hasAccountOrPeriodCue;
|
||||
});
|
||||
}
|
||||
function hasDeepAnalysisPreferenceSignal(text) {
|
||||
const repaired = repairAddressMojibake(String(text ?? ""));
|
||||
const lower = compactWhitespace(repaired.toLowerCase());
|
||||
if (!lower) {
|
||||
return false;
|
||||
}
|
||||
const riskOrAnomalySignal = /(?:\u0440\u0438\u0441\u043a|risk|\u0430\u043d\u043e\u043c\u0430\u043b|anomal|\u043f\u0440\u043e\u0442\u0438\u0432\u043e\u0440\u0435\u0447|\u043a\u043e\u043d\u0444\u043b\u0438\u043a\u0442|conflict|deviation|\u043e\u0442\u043a\u043b\u043e\u043d\u0435\u043d|\u043d\u0435\u0441\u044b\u043a\u043e\u0432\u043a|\u043d\u0435\u0441\u0445\u043e\u0434|\u043e\u0448\u0438\u0431|error|issue|\u043f\u0440\u043e\u0431\u043b\u0435\u043c)/iu.test(lower);
|
||||
const chainSignal = /(?:\u0446\u0435\u043f\u043e\u0447\u043a|chain|trace\s*chain|lifecycle|\u0436\u0438\u0437\u043d\u0435\u043d\u043d[\u0430-\u044f]+\s+\u0446\u0438\u043a\u043b|state\s+transition|\u0440\u0430\u0437\u0440\u044b\u0432[\u0430-\u044f]*)/iu.test(lower);
|
||||
const diagnosticsSignal = /(?:\u0440\u0430\u0437\u043b\u043e\u0436\u0438|\u0434\u0435\u043a\u043e\u043c\u043f\u043e\u0437|\u0440\u0430\u0437\u0431\u0435\u0440\u0438|\u043f\u043e\u0447\u0435\u043c\u0443|why|\u043a\u043e\u0440\u043d\u0435\u0432[\u0430-\u044f]+\s+\u043f\u0440\u0438\u0447\u0438\u043d|root\s*cause|\u043c\u0435\u0445\u0430\u043d\u0438\u0437\u043c[\u0430-\u044f]*|\u0433\u0434\u0435\s+\u0440\u0430\u0437\u0440\u044b\u0432|\u0447\u0442\u043e\s+\u043c\u0435\u0448\u0430[\u0430-\u044f]+\s+\u0437\u0430\u043a\u0440\u044b\u0442)/iu.test(lower);
|
||||
const closureSignal = /(?:\u0437\u0430\u043a\u0440\u044b\u0442\u0438[\u0435\u044f]\s+\u043f\u0435\u0440\u0438\u043e\u0434|period\s*close|\u043d\u0435\s+\u0437\u0430\u043a\u0440\u044b\u043b[\u0430-\u044f]*|\u0445\u0432\u043e\u0441\u0442[\u0430-\u044f]*)/iu.test(lower);
|
||||
const closureIntentSignal = /(?:\u0437\u0430\u043a\u0440\u044b\u0442[\u0430-\u044f]*|period\s*close|close\s+period)/iu.test(lower);
|
||||
const closureDiagnosticPhraseSignal = /(?:\u0447\u0442\u043e(?:\s+\S+){0,8}\s+\u043c\u0435\u0448\u0430[\u0430-\u044f]+\s+\u0437\u0430\u043a\u0440\u044b\u0442)/iu.test(lower);
|
||||
const signalVsNoiseDiagnostic = /(?:\u043d\u0435\s+\u043f\u0440\u043e\u0441\u0442\u043e\s+(?:\u043d\u0430\s+)?\u0448\u0443\u043c|\u043f\u043e\u0445\u043e\u0436[\u0438\u0435]\s+(?:\u0438\u043c\u0435\u043d\u043d\u043e\s+)?\u043d\u0430\s+\u043f\u0440\u043e\u0431\u043b\u0435\u043c)/iu.test(lower);
|
||||
const lifecycleMismatchSignal = /(?:\u043d\u0435\s+\u0442\u0435\u043c\s+\u0442\u0438\u043f(?:\u043e\u043c)?\s+\u0434\u043e\u043a\u0443\u043c\u0435\u043d\u0442|\u043e\u0436\u0438\u0434\u0430\u0435\u043c[\u0430-\u044f]+\s+\u043f\u0435\u0440\u0435\u0445\u043e\u0434[\u0430-\u044f]*\s+\u043d\u0435\s+\u043f\u043e\u0434\u0442\u0432\u0435\u0440\u0436\u0434|\u043f\u0435\u0440\u0435\u0445\u043e\u0434[\u0430-\u044f]*\s+\u043d\u0435\s+\u043f\u043e\u0434\u0442\u0432\u0435\u0440\u0436\u0434|wrong\s+closing\s+document|expected\s+transition)/iu.test(lower);
|
||||
const lifecycleTransitionGapSignal = /(?:\u043e\u0436\u0438\u0434\u0430\u0435\u043c[\u0430-\u044f]+\s+\u043f\u0435\u0440\u0435\u0445\u043e\u0434[\u0430-\u044f]*\s+\u043e\u0442\u0441\u0443\u0442\u0441\u0442\u0432|\u043f\u0435\u0440\u0435\u0445\u043e\u0434[\u0430-\u044f]*\s+\u043e\u0442\u0441\u0443\u0442\u0441\u0442\u0432|\u0441\u0442\u0430\u0434\u0438[\u0438\u044f\u0435]\s+.*\u043f\u0440\u043e\u0439\u0434\u0435\u043d.*\u043f\u0435\u0440\u0435\u0445\u043e\u0434)/iu.test(lower);
|
||||
const expectedActualMismatchSignal = /(?:\u0444\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a[\u0430-\u044f]+\s+\u0441\u043e\u0441\u0442\u043e\u044f\u043d[\u0438\u0435\u044f]+\s+.*\u0440\u0430\u0441\u0445\u043e\u0434[\u0430-\u044f]*\s+\u0441\s+\u043e\u0436\u0438\u0434\u0430\u0435\u043c|\u043e\u0436\u0438\u0434\u0430\u0435\u043c[\u0430-\u044f]+\s+\u043f\u043e\u0441\u043b\u0435\u0434\u043e\u0432\u0430\u0442\u0435\u043b\u044c\u043d[\u0430-\u044f]*\s+\u0441\u043f\u0438\u0441\u0430\u043d)/iu.test(lower);
|
||||
return riskOrAnomalySignal ||
|
||||
lifecycleMismatchSignal ||
|
||||
(chainSignal && lifecycleTransitionGapSignal) ||
|
||||
expectedActualMismatchSignal ||
|
||||
(chainSignal && diagnosticsSignal) ||
|
||||
(riskOrAnomalySignal && (chainSignal || closureSignal || diagnosticsSignal || closureIntentSignal)) ||
|
||||
(diagnosticsSignal && closureIntentSignal) ||
|
||||
closureDiagnosticPhraseSignal ||
|
||||
signalVsNoiseDiagnostic;
|
||||
}
|
||||
function hasDirectDeepAnalysisSignal(text) {
|
||||
const normalized = compactWhitespace(repairAddressMojibake(String(text ?? "")).toLowerCase());
|
||||
if (!normalized) {
|
||||
return false;
|
||||
}
|
||||
return /(?:\u0440\u0430\u0437\u043b\u043e\u0436|\u0446\u0435\u043f\u043e\u0447|lifecycle|\u0440\u0430\u0437\u0440\u044b\u0432|\u043f\u0440\u043e\u0442\u0438\u0432\u043e\u0440\u0435\u0447|\u0430\u043d\u043e\u043c\u0430\u043b|\u043f\u043e\u0447\u0435\u043c\u0443|\u043c\u0435\u0445\u0430\u043d\u0438\u0437\u043c|\u0437\u0430\u043a\u0440\u044b\u0442[\u0430-\u044f]*|state\s+transition|root\s*cause|trace\s*chain)/iu.test(normalized);
|
||||
}
|
||||
export function resolveAssistantOrchestrationDecision(input) {
|
||||
const rawUserMessage = String(input?.rawUserMessage ?? input?.userMessage ?? "");
|
||||
const effectiveAddressUserMessage = String(input?.effectiveAddressUserMessage ?? rawUserMessage);
|
||||
|
|
@ -3044,6 +3141,7 @@ export function resolveAssistantOrchestrationDecision(input) {
|
|||
const followupContext = input?.followupContext ?? null;
|
||||
const llmPreDecomposeMeta = input?.llmPreDecomposeMeta ?? null;
|
||||
const useMock = Boolean(input?.useMock);
|
||||
const sessionItems = Array.isArray(input?.sessionItems) ? input.sessionItems : null;
|
||||
const dataScopeMetaQuery = hasAssistantDataScopeMetaQuestionSignal(rawUserMessage) ||
|
||||
hasAssistantDataScopeMetaQuestionSignal(repairedRawUserMessage) ||
|
||||
hasAssistantDataScopeMetaQuestionSignal(effectiveAddressUserMessage) ||
|
||||
|
|
@ -3131,11 +3229,52 @@ export function resolveAssistantOrchestrationDecision(input) {
|
|||
};
|
||||
}
|
||||
const baseToolGate = resolveAddressToolGateDecision(effectiveAddressUserMessage, followupContext, llmPreDecomposeMeta, rawUserMessage);
|
||||
const llmContractMode = toNonEmptyString(llmPreDecomposeMeta?.predecomposeContract?.mode);
|
||||
const preserveAddressLaneSignal = Boolean((llmPreDecomposeMeta?.llmCanonicalCandidateDetected &&
|
||||
llmPreDecomposeMeta?.applied &&
|
||||
(llmContractMode === "address_query" || llmContractMode === "unsupported")) ||
|
||||
hasSameDateAccountFollowupSignalForPredecompose(rawUserMessage) ||
|
||||
hasSameDateAccountFollowupSignalForPredecompose(effectiveAddressUserMessage) ||
|
||||
hasSameDateAccountFollowupSignalForPredecompose(repairedRawUserMessage) ||
|
||||
hasSameDateAccountFollowupSignalForPredecompose(repairedEffectiveAddressUserMessage) ||
|
||||
hasLooseAllTimeAddressLookupSignal(rawUserMessage) ||
|
||||
hasLooseAllTimeAddressLookupSignal(effectiveAddressUserMessage) ||
|
||||
hasLooseAllTimeAddressLookupSignal(repairedRawUserMessage) ||
|
||||
hasLooseAllTimeAddressLookupSignal(repairedEffectiveAddressUserMessage) ||
|
||||
hasAddressFollowupContextSignal(rawUserMessage) ||
|
||||
hasAddressFollowupContextSignal(effectiveAddressUserMessage) ||
|
||||
hasAddressFollowupContextSignal(repairedRawUserMessage) ||
|
||||
hasAddressFollowupContextSignal(repairedEffectiveAddressUserMessage));
|
||||
const unsupportedAddressIntentFallbackToDeep = Boolean(!followupContext &&
|
||||
baseToolGate?.runAddressLane &&
|
||||
modeDetection.mode !== "address_query" &&
|
||||
intentResolution.intent === "unknown" &&
|
||||
strongDataSignal);
|
||||
strongDataSignal &&
|
||||
!preserveAddressLaneSignal);
|
||||
const deepAnalysisPreferenceDetected = Boolean(hasDeepAnalysisPreferenceSignal(rawUserMessage) ||
|
||||
hasDeepAnalysisPreferenceSignal(repairedRawUserMessage) ||
|
||||
hasDeepAnalysisPreferenceSignal(effectiveAddressUserMessage) ||
|
||||
hasDeepAnalysisPreferenceSignal(repairedEffectiveAddressUserMessage) ||
|
||||
hasDirectDeepAnalysisSignal(rawUserMessage) ||
|
||||
hasDirectDeepAnalysisSignal(repairedRawUserMessage) ||
|
||||
hasDirectDeepAnalysisSignal(effectiveAddressUserMessage) ||
|
||||
hasDirectDeepAnalysisSignal(repairedEffectiveAddressUserMessage));
|
||||
const vatExplainFollowupSignal = Boolean(followupContext &&
|
||||
toNonEmptyString(followupContext.previous_intent) === "vat_payable_forecast" &&
|
||||
/(?:\u043f\u043e\u0447\u0435\u043c\u0443|why).*(?:\u043f\u0440\u043e\u0433\u043d\u043e\u0437|forecast).*(?:\u0443\u043f\u043b\u0430\u0442|payable|\b0\b)/iu.test(compactWhitespace(`${repairedRawUserMessage} ${repairedEffectiveAddressUserMessage}`)));
|
||||
const deepAnalysisSignalFallbackToDeep = Boolean(baseToolGate?.runAddressLane &&
|
||||
deepAnalysisPreferenceDetected &&
|
||||
!vatExplainFollowupSignal &&
|
||||
(!followupContext || !dataRetrievalSignal));
|
||||
const deepSessionContinuationFallbackToDeep = Boolean(!followupContext &&
|
||||
baseToolGate?.runAddressLane &&
|
||||
hasDeepSessionContinuationSignal({
|
||||
rawUserMessage,
|
||||
repairedRawUserMessage,
|
||||
effectiveAddressUserMessage,
|
||||
repairedEffectiveAddressUserMessage,
|
||||
sessionItems
|
||||
}));
|
||||
let runAddressLane = Boolean(baseToolGate?.runAddressLane);
|
||||
let toolGateDecision = String(baseToolGate?.decision ?? "skip_address_lane");
|
||||
let toolGateReason = String(baseToolGate?.reason ?? "no_address_signal_after_l0");
|
||||
|
|
@ -3144,6 +3283,16 @@ export function resolveAssistantOrchestrationDecision(input) {
|
|||
toolGateDecision = "skip_address_lane";
|
||||
toolGateReason = "address_signal_unsupported_intent_fallback_to_deep";
|
||||
}
|
||||
if (deepAnalysisSignalFallbackToDeep && !unsupportedAddressIntentFallbackToDeep) {
|
||||
runAddressLane = false;
|
||||
toolGateDecision = "skip_address_lane";
|
||||
toolGateReason = "deep_analysis_signal_fallback_to_deep";
|
||||
}
|
||||
if (deepSessionContinuationFallbackToDeep) {
|
||||
runAddressLane = false;
|
||||
toolGateDecision = "skip_address_lane";
|
||||
toolGateReason = "deep_session_continuation_fallback_to_deep";
|
||||
}
|
||||
let livingDecision = resolveLivingAssistantModeDecision({
|
||||
userMessage: rawUserMessage,
|
||||
addressLaneTriggered: runAddressLane,
|
||||
|
|
@ -3157,6 +3306,18 @@ export function resolveAssistantOrchestrationDecision(input) {
|
|||
reason: "unsupported_address_intent_fallback_to_deep"
|
||||
};
|
||||
}
|
||||
if (deepAnalysisSignalFallbackToDeep && !unsupportedAddressIntentFallbackToDeep) {
|
||||
livingDecision = {
|
||||
mode: "deep_analysis",
|
||||
reason: "deep_analysis_signal_fallback_to_deep"
|
||||
};
|
||||
}
|
||||
if (deepSessionContinuationFallbackToDeep) {
|
||||
livingDecision = {
|
||||
mode: "deep_analysis",
|
||||
reason: "deep_session_continuation_fallback_to_deep"
|
||||
};
|
||||
}
|
||||
return {
|
||||
runAddressLane,
|
||||
toolGateDecision,
|
||||
|
|
@ -3174,6 +3335,8 @@ export function resolveAssistantOrchestrationDecision(input) {
|
|||
data_retrieval_signal_detected: dataRetrievalSignal,
|
||||
followup_context_detected: Boolean(followupContext),
|
||||
unsupported_address_intent_fallback_to_deep: unsupportedAddressIntentFallbackToDeep,
|
||||
deep_analysis_signal_fallback_to_deep: deepAnalysisSignalFallbackToDeep,
|
||||
deep_session_continuation_fallback_to_deep: deepSessionContinuationFallbackToDeep,
|
||||
final_decision: {
|
||||
run_address_lane: runAddressLane,
|
||||
tool_gate_decision: toolGateDecision,
|
||||
|
|
|
|||
|
|
@ -85,12 +85,12 @@ export function buildAssistantTurnRuntimeDeps(
|
|||
): AssistantTurnRuntimeBuilderDeps {
|
||||
return {
|
||||
...input.helpers,
|
||||
ensureSession: input.sessions.ensureSession,
|
||||
appendItem: input.sessions.appendItem,
|
||||
getSession: input.sessions.getSession,
|
||||
persistSession: input.sessionLogger.persistSession,
|
||||
setInvestigationState: input.sessions.setInvestigationState,
|
||||
normalize: input.normalizerService.normalize,
|
||||
ensureSession: (sessionId) => input.sessions.ensureSession(sessionId),
|
||||
appendItem: (sessionId, item) => input.sessions.appendItem(sessionId, item),
|
||||
getSession: (sessionId) => input.sessions.getSession(sessionId),
|
||||
persistSession: (session) => input.sessionLogger.persistSession(session),
|
||||
setInvestigationState: (sessionId, state) => input.sessions.setInvestigationState(sessionId, state),
|
||||
normalize: (payload) => input.normalizerService.normalize(payload),
|
||||
executeRouteRuntime: (route, fragmentText, options) =>
|
||||
input.dataLayer.executeRouteRuntime(route, fragmentText, options),
|
||||
tryAddressQueryHandle: (messageUsed, options) => input.addressQueryService.tryHandle(messageUsed, options),
|
||||
|
|
|
|||
|
|
@ -1,4 +1,13 @@
|
|||
import type { NormalizeRequestPayload, NormalizeResponsePayload, RouteHintSummary } from "./normalizer";
|
||||
import type {
|
||||
NormalizeRequestPayload,
|
||||
NormalizeResponsePayload,
|
||||
NoRouteReason,
|
||||
RouteHintSummary,
|
||||
RouteStatus,
|
||||
ExecutionReadiness,
|
||||
ConfidenceLevel,
|
||||
IntentClass
|
||||
} from "./normalizer";
|
||||
import type { AnswerStructureV11, EvidenceItem } from "./stage1Contracts";
|
||||
import type {
|
||||
CandidateEvidenceItem,
|
||||
|
|
@ -27,6 +36,43 @@ export type AssistantProblemAnswerMode =
|
|||
| "stage2_problem_centric_v1"
|
||||
| "stage3_lifecycle_aware_v1";
|
||||
|
||||
export interface AssistantExecutionStateRecord {
|
||||
fragment_id: string | null;
|
||||
execution_readiness: ExecutionReadiness | null;
|
||||
route_status: RouteStatus | null;
|
||||
no_route_reason: NoRouteReason | null;
|
||||
}
|
||||
|
||||
export type AssistantDebugRouteRecord =
|
||||
| {
|
||||
fragment_id: string;
|
||||
route: string;
|
||||
reason: string;
|
||||
confidence: ConfidenceLevel;
|
||||
intent_class: IntentClass;
|
||||
}
|
||||
| {
|
||||
fragment_id: string;
|
||||
route: string;
|
||||
reason: string;
|
||||
route_status: RouteStatus | null;
|
||||
no_route_reason: NoRouteReason | null;
|
||||
clarification_reason: string | null;
|
||||
execution_readiness: ExecutionReadiness | null;
|
||||
};
|
||||
|
||||
export interface AssistantAddressRuntimeMetaForDeep {
|
||||
attempted?: boolean;
|
||||
applied?: boolean;
|
||||
reason?: string | null;
|
||||
provider?: string | null;
|
||||
fallbackRuleHit?: string | null;
|
||||
toolGateDecision?: string | null;
|
||||
toolGateReason?: string | null;
|
||||
predecomposeContract?: Record<string, unknown> | null;
|
||||
orchestrationContract?: Record<string, unknown> | null;
|
||||
}
|
||||
|
||||
export interface AssistantRequirement {
|
||||
requirement_id: string;
|
||||
source_fragment_id: string | null;
|
||||
|
|
@ -302,7 +348,7 @@ export interface AssistantDebugPayload {
|
|||
fragments: unknown[];
|
||||
requirements_extracted: AssistantRequirement[];
|
||||
coverage_report: RequirementCoverageReport;
|
||||
routes: Array<Record<string, unknown>>;
|
||||
routes: AssistantDebugRouteRecord[];
|
||||
retrieval_status: Array<{
|
||||
fragment_id: string;
|
||||
requirement_ids: string[];
|
||||
|
|
|
|||
|
|
@ -32,6 +32,9 @@ function redactObject(value: unknown): unknown {
|
|||
}
|
||||
|
||||
export function logJson(entry: JsonLogEntry): void {
|
||||
if (process.env.NODE_ENV === "test" && process.env.FEATURE_JSON_STDOUT_LOGS_IN_TESTS !== "1") {
|
||||
return;
|
||||
}
|
||||
const safe = {
|
||||
...entry,
|
||||
details: redactObject(entry.details)
|
||||
|
|
|
|||
|
|
@ -261,4 +261,40 @@ describe("assistant orchestration contract", () => {
|
|||
decision.toolGateReason
|
||||
);
|
||||
});
|
||||
|
||||
it("routes risk/anomaly analytics wording to deep pipeline", () => {
|
||||
const decision = resolveAssistantOrchestrationDecision({
|
||||
rawUserMessage: "Проверь НДС по счету 19 за 2020-06 и рискованные записи по документам.",
|
||||
effectiveAddressUserMessage: "Проверь НДС по счету 19 за 2020-06 и рискованные записи по документам.",
|
||||
followupContext: null,
|
||||
llmPreDecomposeMeta: null,
|
||||
useMock: false
|
||||
});
|
||||
|
||||
expect(decision.runAddressLane).toBe(false);
|
||||
expect(decision.toolGateDecision).toBe("skip_address_lane");
|
||||
expect(decision.toolGateReason).toBe("deep_analysis_signal_fallback_to_deep");
|
||||
expect(decision.livingMode).toBe("deep_analysis");
|
||||
expect(decision.livingReason).toBe("deep_analysis_signal_fallback_to_deep");
|
||||
expect(decision.orchestrationContract?.deep_analysis_signal_fallback_to_deep).toBe(true);
|
||||
});
|
||||
|
||||
it("routes settlement closure verification wording to deep pipeline", () => {
|
||||
const decision = resolveAssistantOrchestrationDecision({
|
||||
rawUserMessage:
|
||||
"По оплате поставщику на счете 60 в июле 2020 остался хвост. Проверь закрытие по договору и объекту расчетов.",
|
||||
effectiveAddressUserMessage:
|
||||
"По оплате поставщику на счете 60 в июле 2020 остался хвост. Проверь закрытие по договору и объекту расчетов.",
|
||||
followupContext: null,
|
||||
llmPreDecomposeMeta: null,
|
||||
useMock: false
|
||||
});
|
||||
|
||||
expect(decision.runAddressLane).toBe(false);
|
||||
expect(decision.toolGateDecision).toBe("skip_address_lane");
|
||||
expect(decision.toolGateReason).toBe("deep_analysis_signal_fallback_to_deep");
|
||||
expect(decision.livingMode).toBe("deep_analysis");
|
||||
expect(decision.livingReason).toBe("deep_analysis_signal_fallback_to_deep");
|
||||
expect(decision.orchestrationContract?.deep_analysis_signal_fallback_to_deep).toBe(true);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -129,4 +129,89 @@ describe("assistant turn runtime deps adapter", () => {
|
|||
expect(deps.compactWhitespace(" value ")).toBe("value");
|
||||
expect(helperCompactWhitespace).toHaveBeenCalledWith(" value ");
|
||||
});
|
||||
|
||||
it("preserves method context for stateful service instances", async () => {
|
||||
class SessionStoreLike {
|
||||
public calls: string[] = [];
|
||||
public ensureSession(sessionId: string) {
|
||||
this.calls.push(`ensure:${sessionId}`);
|
||||
return { session_id: sessionId } as any;
|
||||
}
|
||||
public appendItem(sessionId: string) {
|
||||
this.calls.push(`append:${sessionId}`);
|
||||
}
|
||||
public getSession(sessionId: string) {
|
||||
this.calls.push(`get:${sessionId}`);
|
||||
return null;
|
||||
}
|
||||
public setInvestigationState(sessionId: string) {
|
||||
this.calls.push(`state:${sessionId}`);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
class SessionLoggerLike {
|
||||
public persisted = 0;
|
||||
public persistSession() {
|
||||
this.persisted += 1;
|
||||
}
|
||||
}
|
||||
|
||||
class NormalizerLike {
|
||||
public normalized = 0;
|
||||
public async normalize() {
|
||||
this.normalized += 1;
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
const sessions = new SessionStoreLike();
|
||||
const sessionLogger = new SessionLoggerLike();
|
||||
const normalizerService = new NormalizerLike();
|
||||
|
||||
const deps = buildAssistantTurnRuntimeDeps({
|
||||
sessions,
|
||||
sessionLogger,
|
||||
normalizerService,
|
||||
dataLayer: {
|
||||
executeRouteRuntime: async () => ({})
|
||||
},
|
||||
addressQueryService: {
|
||||
tryHandle: async () => null
|
||||
},
|
||||
chatClient: {},
|
||||
messageIdFactory: () => "msg-ctx",
|
||||
nowIso: () => "2026-04-11T00:00:00.000Z",
|
||||
defaultApiKey: "",
|
||||
logEvent: () => {},
|
||||
flags: {
|
||||
featureAssistantAddressQueryV1: false,
|
||||
featureAddressLlmPredecomposeV1: false,
|
||||
featureInvestigationStateV1: false,
|
||||
featureStateFollowupBindingV1: false,
|
||||
featureContractsV11: false,
|
||||
featureAnswerPolicyV11: false,
|
||||
featureProblemCentricAnswerV1: false,
|
||||
featureLifecycleAnswerV1: false
|
||||
},
|
||||
defaults: {
|
||||
defaultModel: "model",
|
||||
defaultBaseUrl: "base"
|
||||
},
|
||||
helpers: {
|
||||
compactWhitespace: (value: unknown) => String(value ?? "")
|
||||
} as any
|
||||
});
|
||||
|
||||
deps.ensureSession("asst-ctx");
|
||||
deps.appendItem("asst-ctx", { role: "user" } as any);
|
||||
deps.getSession("asst-ctx");
|
||||
deps.setInvestigationState("asst-ctx", null);
|
||||
deps.persistSession({ session_id: "asst-ctx" } as any);
|
||||
await deps.normalize({ user_message: "ctx" } as any);
|
||||
|
||||
expect(sessions.calls).toEqual(["ensure:asst-ctx", "append:asst-ctx", "get:asst-ctx", "state:asst-ctx"]);
|
||||
expect(sessionLogger.persisted).toBe(1);
|
||||
expect(normalizerService.normalized).toBe(1);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,112 @@
|
|||
{
|
||||
"run_id": "eval-5S6BYeWJWv",
|
||||
"timestamp": "2026-04-11T10:38:19.513Z",
|
||||
"mode": "single-pass-strict",
|
||||
"use_mock": true,
|
||||
"prompt_version": "normalizer_v2_0_2",
|
||||
"schema_version": "v2_0_2",
|
||||
"dataset": {
|
||||
"source": "inline_raw_questions",
|
||||
"file": null,
|
||||
"raw_questions_count": 2
|
||||
},
|
||||
"cases_total": 2,
|
||||
"metrics": {
|
||||
"schema_validation_pass_rate": 100,
|
||||
"scope_detection_accuracy": null,
|
||||
"scope_in_scope_rate": 100,
|
||||
"multi_intent_detected_rate": 0,
|
||||
"clarification_required_rate": 0,
|
||||
"avg_fragments_per_message": 1,
|
||||
"out_of_scope_fragment_rate": 0,
|
||||
"routed_fragment_rate": 100,
|
||||
"no_route_fragment_rate": 0,
|
||||
"route_resolution_accuracy": null,
|
||||
"no_route_precision": null,
|
||||
"false_no_route_rate": null,
|
||||
"execution_state_consistency_rate": 100,
|
||||
"executable_with_soft_assumptions_rate": 100,
|
||||
"soft_assumption_used_fragment_rate": 100,
|
||||
"clarification_precision": null,
|
||||
"clarification_recall": null,
|
||||
"false_clarification_rate": null
|
||||
},
|
||||
"budget": {
|
||||
"requests_total": 0,
|
||||
"retries_used": 0
|
||||
},
|
||||
"clarification_eval": {
|
||||
"labeled_cases": 0,
|
||||
"true_positive": 0,
|
||||
"false_positive": 0,
|
||||
"false_negative": 0
|
||||
},
|
||||
"route_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0,
|
||||
"expected_routed_cases": 0,
|
||||
"no_route_true_positive": 0,
|
||||
"no_route_false_positive": 0
|
||||
},
|
||||
"scope_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0
|
||||
},
|
||||
"execution_state_eval": {
|
||||
"checks_total": 2,
|
||||
"checks_passed": 2
|
||||
},
|
||||
"route_distribution": {
|
||||
"store_feature_risk": 1,
|
||||
"hybrid_store_plus_live": 1
|
||||
},
|
||||
"fallback_distribution": {
|
||||
"none": 2
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"case_id": "BQ-001",
|
||||
"raw_question": "Проверь счет 60 за июнь 2020",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "bi53megxPkVDpq",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-002",
|
||||
"raw_question": "Покажи риски по НДС и по закрытию",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "ew725J5aRLs1XU",
|
||||
"request_count_for_case": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
{
|
||||
"run_id": "eval-D1gt1OyBqh",
|
||||
"timestamp": "2026-04-11T10:35:37.557Z",
|
||||
"mode": "single-pass-strict",
|
||||
"use_mock": true,
|
||||
"prompt_version": "normalizer_v2_0_2",
|
||||
"schema_version": "v2_0_2",
|
||||
"dataset": {
|
||||
"source": "inline_raw_questions",
|
||||
"file": null,
|
||||
"raw_questions_count": 2
|
||||
},
|
||||
"cases_total": 2,
|
||||
"metrics": {
|
||||
"schema_validation_pass_rate": 100,
|
||||
"scope_detection_accuracy": null,
|
||||
"scope_in_scope_rate": 100,
|
||||
"multi_intent_detected_rate": 0,
|
||||
"clarification_required_rate": 0,
|
||||
"avg_fragments_per_message": 1,
|
||||
"out_of_scope_fragment_rate": 0,
|
||||
"routed_fragment_rate": 100,
|
||||
"no_route_fragment_rate": 0,
|
||||
"route_resolution_accuracy": null,
|
||||
"no_route_precision": null,
|
||||
"false_no_route_rate": null,
|
||||
"execution_state_consistency_rate": 100,
|
||||
"executable_with_soft_assumptions_rate": 100,
|
||||
"soft_assumption_used_fragment_rate": 100,
|
||||
"clarification_precision": null,
|
||||
"clarification_recall": null,
|
||||
"false_clarification_rate": null
|
||||
},
|
||||
"budget": {
|
||||
"requests_total": 0,
|
||||
"retries_used": 0
|
||||
},
|
||||
"clarification_eval": {
|
||||
"labeled_cases": 0,
|
||||
"true_positive": 0,
|
||||
"false_positive": 0,
|
||||
"false_negative": 0
|
||||
},
|
||||
"route_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0,
|
||||
"expected_routed_cases": 0,
|
||||
"no_route_true_positive": 0,
|
||||
"no_route_false_positive": 0
|
||||
},
|
||||
"scope_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0
|
||||
},
|
||||
"execution_state_eval": {
|
||||
"checks_total": 2,
|
||||
"checks_passed": 2
|
||||
},
|
||||
"route_distribution": {
|
||||
"store_feature_risk": 1,
|
||||
"hybrid_store_plus_live": 1
|
||||
},
|
||||
"fallback_distribution": {
|
||||
"none": 2
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"case_id": "BQ-001",
|
||||
"raw_question": "Проверь счет 60 за июнь 2020",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "0XdpLLt2DVEC8b",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-002",
|
||||
"raw_question": "Покажи риски по НДС и по закрытию",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "TGXTmJEIbfRfxl",
|
||||
"request_count_for_case": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,137 @@
|
|||
{
|
||||
"run_id": "eval-EYDYh4k78m",
|
||||
"timestamp": "2026-04-11T10:35:23.070Z",
|
||||
"mode": "single-pass-strict",
|
||||
"use_mock": true,
|
||||
"prompt_version": "normalizer_v2_0_2",
|
||||
"schema_version": "v2_0_2",
|
||||
"dataset": {
|
||||
"source": "inline_raw_questions",
|
||||
"file": null,
|
||||
"raw_questions_count": 3
|
||||
},
|
||||
"cases_total": 3,
|
||||
"metrics": {
|
||||
"schema_validation_pass_rate": 100,
|
||||
"scope_detection_accuracy": null,
|
||||
"scope_in_scope_rate": 33.33,
|
||||
"multi_intent_detected_rate": 0,
|
||||
"clarification_required_rate": 0,
|
||||
"avg_fragments_per_message": 1,
|
||||
"out_of_scope_fragment_rate": 33.33,
|
||||
"routed_fragment_rate": 66.67,
|
||||
"no_route_fragment_rate": 33.33,
|
||||
"route_resolution_accuracy": null,
|
||||
"no_route_precision": null,
|
||||
"false_no_route_rate": null,
|
||||
"execution_state_consistency_rate": 66.67,
|
||||
"executable_with_soft_assumptions_rate": 100,
|
||||
"soft_assumption_used_fragment_rate": 100,
|
||||
"clarification_precision": null,
|
||||
"clarification_recall": null,
|
||||
"false_clarification_rate": null
|
||||
},
|
||||
"budget": {
|
||||
"requests_total": 0,
|
||||
"retries_used": 0
|
||||
},
|
||||
"clarification_eval": {
|
||||
"labeled_cases": 0,
|
||||
"true_positive": 0,
|
||||
"false_positive": 0,
|
||||
"false_negative": 0
|
||||
},
|
||||
"route_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0,
|
||||
"expected_routed_cases": 0,
|
||||
"no_route_true_positive": 0,
|
||||
"no_route_false_positive": 0
|
||||
},
|
||||
"scope_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0
|
||||
},
|
||||
"execution_state_eval": {
|
||||
"checks_total": 3,
|
||||
"checks_passed": 2
|
||||
},
|
||||
"route_distribution": {
|
||||
"hybrid_store_plus_live": 1,
|
||||
"no_route": 1,
|
||||
"batch_refresh_then_store": 1
|
||||
},
|
||||
"fallback_distribution": {
|
||||
"none": 1,
|
||||
"out_of_scope": 1,
|
||||
"clarification": 1
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"case_id": "BQ-001",
|
||||
"raw_question": "Проверь хвосты по поставщикам и разложи цепочку",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "XkM4LP-bXu4rf9",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-002",
|
||||
"raw_question": "Как вообще по ФСБУ",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": false,
|
||||
"scope_confidence": "low",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 0,
|
||||
"out_of_scope_fragments": 1,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "out_of_scope",
|
||||
"predicted_route_status": "no_route",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": "out_of_scope",
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 0,
|
||||
"trace_id": "eDF6_4JOsguyhk",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-003",
|
||||
"raw_question": "Покажи топ рисков за июнь 2020",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": false,
|
||||
"scope_confidence": "low",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 0,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 1,
|
||||
"fallback_type": "clarification",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 0,
|
||||
"trace_id": "dsCVkLkLYqwqHr",
|
||||
"request_count_for_case": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
{
|
||||
"run_id": "eval-NeqPDTflc7",
|
||||
"timestamp": "2026-04-11T10:16:53.909Z",
|
||||
"mode": "single-pass-strict",
|
||||
"use_mock": true,
|
||||
"prompt_version": "normalizer_v2_0_2",
|
||||
"schema_version": "v2_0_2",
|
||||
"dataset": {
|
||||
"source": "inline_raw_questions",
|
||||
"file": null,
|
||||
"raw_questions_count": 2
|
||||
},
|
||||
"cases_total": 2,
|
||||
"metrics": {
|
||||
"schema_validation_pass_rate": 100,
|
||||
"scope_detection_accuracy": null,
|
||||
"scope_in_scope_rate": 100,
|
||||
"multi_intent_detected_rate": 0,
|
||||
"clarification_required_rate": 0,
|
||||
"avg_fragments_per_message": 1,
|
||||
"out_of_scope_fragment_rate": 0,
|
||||
"routed_fragment_rate": 100,
|
||||
"no_route_fragment_rate": 0,
|
||||
"route_resolution_accuracy": null,
|
||||
"no_route_precision": null,
|
||||
"false_no_route_rate": null,
|
||||
"execution_state_consistency_rate": 100,
|
||||
"executable_with_soft_assumptions_rate": 100,
|
||||
"soft_assumption_used_fragment_rate": 100,
|
||||
"clarification_precision": null,
|
||||
"clarification_recall": null,
|
||||
"false_clarification_rate": null
|
||||
},
|
||||
"budget": {
|
||||
"requests_total": 0,
|
||||
"retries_used": 0
|
||||
},
|
||||
"clarification_eval": {
|
||||
"labeled_cases": 0,
|
||||
"true_positive": 0,
|
||||
"false_positive": 0,
|
||||
"false_negative": 0
|
||||
},
|
||||
"route_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0,
|
||||
"expected_routed_cases": 0,
|
||||
"no_route_true_positive": 0,
|
||||
"no_route_false_positive": 0
|
||||
},
|
||||
"scope_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0
|
||||
},
|
||||
"execution_state_eval": {
|
||||
"checks_total": 2,
|
||||
"checks_passed": 2
|
||||
},
|
||||
"route_distribution": {
|
||||
"store_feature_risk": 1,
|
||||
"hybrid_store_plus_live": 1
|
||||
},
|
||||
"fallback_distribution": {
|
||||
"none": 2
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"case_id": "BQ-001",
|
||||
"raw_question": "Проверь счет 60 за июнь 2020",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "ynDJfmhtaBS6Jh",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-002",
|
||||
"raw_question": "Покажи риски по счету 97",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "BhT9NfO-40oigp",
|
||||
"request_count_for_case": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,137 @@
|
|||
{
|
||||
"run_id": "eval-OSNnoiu9KQ",
|
||||
"timestamp": "2026-04-11T10:16:34.741Z",
|
||||
"mode": "single-pass-strict",
|
||||
"use_mock": true,
|
||||
"prompt_version": "normalizer_v2_0_2",
|
||||
"schema_version": "v2_0_2",
|
||||
"dataset": {
|
||||
"source": "inline_raw_questions",
|
||||
"file": null,
|
||||
"raw_questions_count": 3
|
||||
},
|
||||
"cases_total": 3,
|
||||
"metrics": {
|
||||
"schema_validation_pass_rate": 100,
|
||||
"scope_detection_accuracy": null,
|
||||
"scope_in_scope_rate": 33.33,
|
||||
"multi_intent_detected_rate": 0,
|
||||
"clarification_required_rate": 0,
|
||||
"avg_fragments_per_message": 1,
|
||||
"out_of_scope_fragment_rate": 33.33,
|
||||
"routed_fragment_rate": 66.67,
|
||||
"no_route_fragment_rate": 33.33,
|
||||
"route_resolution_accuracy": null,
|
||||
"no_route_precision": null,
|
||||
"false_no_route_rate": null,
|
||||
"execution_state_consistency_rate": 66.67,
|
||||
"executable_with_soft_assumptions_rate": 100,
|
||||
"soft_assumption_used_fragment_rate": 100,
|
||||
"clarification_precision": null,
|
||||
"clarification_recall": null,
|
||||
"false_clarification_rate": null
|
||||
},
|
||||
"budget": {
|
||||
"requests_total": 0,
|
||||
"retries_used": 0
|
||||
},
|
||||
"clarification_eval": {
|
||||
"labeled_cases": 0,
|
||||
"true_positive": 0,
|
||||
"false_positive": 0,
|
||||
"false_negative": 0
|
||||
},
|
||||
"route_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0,
|
||||
"expected_routed_cases": 0,
|
||||
"no_route_true_positive": 0,
|
||||
"no_route_false_positive": 0
|
||||
},
|
||||
"scope_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0
|
||||
},
|
||||
"execution_state_eval": {
|
||||
"checks_total": 3,
|
||||
"checks_passed": 2
|
||||
},
|
||||
"route_distribution": {
|
||||
"hybrid_store_plus_live": 1,
|
||||
"no_route": 1,
|
||||
"batch_refresh_then_store": 1
|
||||
},
|
||||
"fallback_distribution": {
|
||||
"none": 1,
|
||||
"out_of_scope": 1,
|
||||
"clarification": 1
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"case_id": "BQ-001",
|
||||
"raw_question": "Проверь хвосты по поставщикам и разложи цепочку",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "YPmWhiLGA674vn",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-002",
|
||||
"raw_question": "Как вообще по ФСБУ",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": false,
|
||||
"scope_confidence": "low",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 0,
|
||||
"out_of_scope_fragments": 1,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "out_of_scope",
|
||||
"predicted_route_status": "no_route",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": "out_of_scope",
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 0,
|
||||
"trace_id": "JAbihOWvYs9bYR",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-003",
|
||||
"raw_question": "Покажи топ рисков за июнь 2020",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": false,
|
||||
"scope_confidence": "low",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 0,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 1,
|
||||
"fallback_type": "clarification",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 0,
|
||||
"trace_id": "WuI3XbextvaDxw",
|
||||
"request_count_for_case": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
{
|
||||
"run_id": "eval-RVeDpGREWm",
|
||||
"timestamp": "2026-04-11T10:14:44.794Z",
|
||||
"mode": "single-pass-strict",
|
||||
"use_mock": true,
|
||||
"prompt_version": "normalizer_v2_0_2",
|
||||
"schema_version": "v2_0_2",
|
||||
"dataset": {
|
||||
"source": "inline_raw_questions",
|
||||
"file": null,
|
||||
"raw_questions_count": 2
|
||||
},
|
||||
"cases_total": 2,
|
||||
"metrics": {
|
||||
"schema_validation_pass_rate": 100,
|
||||
"scope_detection_accuracy": null,
|
||||
"scope_in_scope_rate": 100,
|
||||
"multi_intent_detected_rate": 0,
|
||||
"clarification_required_rate": 0,
|
||||
"avg_fragments_per_message": 1,
|
||||
"out_of_scope_fragment_rate": 0,
|
||||
"routed_fragment_rate": 100,
|
||||
"no_route_fragment_rate": 0,
|
||||
"route_resolution_accuracy": null,
|
||||
"no_route_precision": null,
|
||||
"false_no_route_rate": null,
|
||||
"execution_state_consistency_rate": 100,
|
||||
"executable_with_soft_assumptions_rate": 100,
|
||||
"soft_assumption_used_fragment_rate": 100,
|
||||
"clarification_precision": null,
|
||||
"clarification_recall": null,
|
||||
"false_clarification_rate": null
|
||||
},
|
||||
"budget": {
|
||||
"requests_total": 0,
|
||||
"retries_used": 0
|
||||
},
|
||||
"clarification_eval": {
|
||||
"labeled_cases": 0,
|
||||
"true_positive": 0,
|
||||
"false_positive": 0,
|
||||
"false_negative": 0
|
||||
},
|
||||
"route_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0,
|
||||
"expected_routed_cases": 0,
|
||||
"no_route_true_positive": 0,
|
||||
"no_route_false_positive": 0
|
||||
},
|
||||
"scope_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0
|
||||
},
|
||||
"execution_state_eval": {
|
||||
"checks_total": 2,
|
||||
"checks_passed": 2
|
||||
},
|
||||
"route_distribution": {
|
||||
"store_feature_risk": 1,
|
||||
"hybrid_store_plus_live": 1
|
||||
},
|
||||
"fallback_distribution": {
|
||||
"none": 2
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"case_id": "BQ-001",
|
||||
"raw_question": "Проверь счет 60 за июнь 2020",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "9WDoQo0uvKHsFh",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-002",
|
||||
"raw_question": "Покажи риски по НДС и по закрытию",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "YJ7f6z9_SS7ZgW",
|
||||
"request_count_for_case": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,137 @@
|
|||
{
|
||||
"run_id": "eval-hhXvDmFzGJ",
|
||||
"timestamp": "2026-04-11T10:14:25.501Z",
|
||||
"mode": "single-pass-strict",
|
||||
"use_mock": true,
|
||||
"prompt_version": "normalizer_v2_0_2",
|
||||
"schema_version": "v2_0_2",
|
||||
"dataset": {
|
||||
"source": "inline_raw_questions",
|
||||
"file": null,
|
||||
"raw_questions_count": 3
|
||||
},
|
||||
"cases_total": 3,
|
||||
"metrics": {
|
||||
"schema_validation_pass_rate": 100,
|
||||
"scope_detection_accuracy": null,
|
||||
"scope_in_scope_rate": 33.33,
|
||||
"multi_intent_detected_rate": 0,
|
||||
"clarification_required_rate": 0,
|
||||
"avg_fragments_per_message": 1,
|
||||
"out_of_scope_fragment_rate": 33.33,
|
||||
"routed_fragment_rate": 66.67,
|
||||
"no_route_fragment_rate": 33.33,
|
||||
"route_resolution_accuracy": null,
|
||||
"no_route_precision": null,
|
||||
"false_no_route_rate": null,
|
||||
"execution_state_consistency_rate": 66.67,
|
||||
"executable_with_soft_assumptions_rate": 100,
|
||||
"soft_assumption_used_fragment_rate": 100,
|
||||
"clarification_precision": null,
|
||||
"clarification_recall": null,
|
||||
"false_clarification_rate": null
|
||||
},
|
||||
"budget": {
|
||||
"requests_total": 0,
|
||||
"retries_used": 0
|
||||
},
|
||||
"clarification_eval": {
|
||||
"labeled_cases": 0,
|
||||
"true_positive": 0,
|
||||
"false_positive": 0,
|
||||
"false_negative": 0
|
||||
},
|
||||
"route_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0,
|
||||
"expected_routed_cases": 0,
|
||||
"no_route_true_positive": 0,
|
||||
"no_route_false_positive": 0
|
||||
},
|
||||
"scope_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0
|
||||
},
|
||||
"execution_state_eval": {
|
||||
"checks_total": 3,
|
||||
"checks_passed": 2
|
||||
},
|
||||
"route_distribution": {
|
||||
"hybrid_store_plus_live": 1,
|
||||
"no_route": 1,
|
||||
"batch_refresh_then_store": 1
|
||||
},
|
||||
"fallback_distribution": {
|
||||
"none": 1,
|
||||
"out_of_scope": 1,
|
||||
"clarification": 1
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"case_id": "BQ-001",
|
||||
"raw_question": "Проверь хвосты по поставщикам и разложи цепочку",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "5-MShsMwdLMm1O",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-002",
|
||||
"raw_question": "Как вообще по ФСБУ",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": false,
|
||||
"scope_confidence": "low",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 0,
|
||||
"out_of_scope_fragments": 1,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "out_of_scope",
|
||||
"predicted_route_status": "no_route",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": "out_of_scope",
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 0,
|
||||
"trace_id": "pv-o-6Cock8Ve2",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-003",
|
||||
"raw_question": "Покажи топ рисков за июнь 2020",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": false,
|
||||
"scope_confidence": "low",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 0,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 1,
|
||||
"fallback_type": "clarification",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 0,
|
||||
"trace_id": "K0zAFkzMAu2BqF",
|
||||
"request_count_for_case": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
{
|
||||
"run_id": "eval-kBTYkPR4wq",
|
||||
"timestamp": "2026-04-11T10:16:53.895Z",
|
||||
"mode": "single-pass-strict",
|
||||
"use_mock": true,
|
||||
"prompt_version": "normalizer_v2_0_2",
|
||||
"schema_version": "v2_0_2",
|
||||
"dataset": {
|
||||
"source": "inline_raw_questions",
|
||||
"file": null,
|
||||
"raw_questions_count": 2
|
||||
},
|
||||
"cases_total": 2,
|
||||
"metrics": {
|
||||
"schema_validation_pass_rate": 100,
|
||||
"scope_detection_accuracy": null,
|
||||
"scope_in_scope_rate": 100,
|
||||
"multi_intent_detected_rate": 0,
|
||||
"clarification_required_rate": 0,
|
||||
"avg_fragments_per_message": 1,
|
||||
"out_of_scope_fragment_rate": 0,
|
||||
"routed_fragment_rate": 100,
|
||||
"no_route_fragment_rate": 0,
|
||||
"route_resolution_accuracy": null,
|
||||
"no_route_precision": null,
|
||||
"false_no_route_rate": null,
|
||||
"execution_state_consistency_rate": 100,
|
||||
"executable_with_soft_assumptions_rate": 100,
|
||||
"soft_assumption_used_fragment_rate": 100,
|
||||
"clarification_precision": null,
|
||||
"clarification_recall": null,
|
||||
"false_clarification_rate": null
|
||||
},
|
||||
"budget": {
|
||||
"requests_total": 0,
|
||||
"retries_used": 0
|
||||
},
|
||||
"clarification_eval": {
|
||||
"labeled_cases": 0,
|
||||
"true_positive": 0,
|
||||
"false_positive": 0,
|
||||
"false_negative": 0
|
||||
},
|
||||
"route_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0,
|
||||
"expected_routed_cases": 0,
|
||||
"no_route_true_positive": 0,
|
||||
"no_route_false_positive": 0
|
||||
},
|
||||
"scope_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0
|
||||
},
|
||||
"execution_state_eval": {
|
||||
"checks_total": 2,
|
||||
"checks_passed": 2
|
||||
},
|
||||
"route_distribution": {
|
||||
"store_feature_risk": 1,
|
||||
"hybrid_store_plus_live": 1
|
||||
},
|
||||
"fallback_distribution": {
|
||||
"none": 2
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"case_id": "BQ-001",
|
||||
"raw_question": "Проверь счет 60 за июнь 2020",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "ZQ4J6HgngY7JB4",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-002",
|
||||
"raw_question": "Покажи риски по НДС и по закрытию",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "Vk2rIlI-UGAuAE",
|
||||
"request_count_for_case": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
{
|
||||
"run_id": "eval-nL2h6Z0kKG",
|
||||
"timestamp": "2026-04-11T10:35:37.559Z",
|
||||
"mode": "single-pass-strict",
|
||||
"use_mock": true,
|
||||
"prompt_version": "normalizer_v2_0_2",
|
||||
"schema_version": "v2_0_2",
|
||||
"dataset": {
|
||||
"source": "inline_raw_questions",
|
||||
"file": null,
|
||||
"raw_questions_count": 2
|
||||
},
|
||||
"cases_total": 2,
|
||||
"metrics": {
|
||||
"schema_validation_pass_rate": 100,
|
||||
"scope_detection_accuracy": null,
|
||||
"scope_in_scope_rate": 100,
|
||||
"multi_intent_detected_rate": 0,
|
||||
"clarification_required_rate": 0,
|
||||
"avg_fragments_per_message": 1,
|
||||
"out_of_scope_fragment_rate": 0,
|
||||
"routed_fragment_rate": 100,
|
||||
"no_route_fragment_rate": 0,
|
||||
"route_resolution_accuracy": null,
|
||||
"no_route_precision": null,
|
||||
"false_no_route_rate": null,
|
||||
"execution_state_consistency_rate": 100,
|
||||
"executable_with_soft_assumptions_rate": 100,
|
||||
"soft_assumption_used_fragment_rate": 100,
|
||||
"clarification_precision": null,
|
||||
"clarification_recall": null,
|
||||
"false_clarification_rate": null
|
||||
},
|
||||
"budget": {
|
||||
"requests_total": 0,
|
||||
"retries_used": 0
|
||||
},
|
||||
"clarification_eval": {
|
||||
"labeled_cases": 0,
|
||||
"true_positive": 0,
|
||||
"false_positive": 0,
|
||||
"false_negative": 0
|
||||
},
|
||||
"route_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0,
|
||||
"expected_routed_cases": 0,
|
||||
"no_route_true_positive": 0,
|
||||
"no_route_false_positive": 0
|
||||
},
|
||||
"scope_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0
|
||||
},
|
||||
"execution_state_eval": {
|
||||
"checks_total": 2,
|
||||
"checks_passed": 2
|
||||
},
|
||||
"route_distribution": {
|
||||
"store_feature_risk": 1,
|
||||
"hybrid_store_plus_live": 1
|
||||
},
|
||||
"fallback_distribution": {
|
||||
"none": 2
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"case_id": "BQ-001",
|
||||
"raw_question": "Проверь счет 60 за июнь 2020",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "gKYkJDhEWkCQTw",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-002",
|
||||
"raw_question": "Покажи риски по счету 97",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "YDK8VaP88Tx8dy",
|
||||
"request_count_for_case": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
{
|
||||
"run_id": "eval-qYzl2S0IV9",
|
||||
"timestamp": "2026-04-11T10:37:58.754Z",
|
||||
"mode": "single-pass-strict",
|
||||
"use_mock": true,
|
||||
"prompt_version": "normalizer_v2_0_2",
|
||||
"schema_version": "v2_0_2",
|
||||
"dataset": {
|
||||
"source": "inline_raw_questions",
|
||||
"file": null,
|
||||
"raw_questions_count": 2
|
||||
},
|
||||
"cases_total": 2,
|
||||
"metrics": {
|
||||
"schema_validation_pass_rate": 100,
|
||||
"scope_detection_accuracy": null,
|
||||
"scope_in_scope_rate": 100,
|
||||
"multi_intent_detected_rate": 0,
|
||||
"clarification_required_rate": 0,
|
||||
"avg_fragments_per_message": 1,
|
||||
"out_of_scope_fragment_rate": 0,
|
||||
"routed_fragment_rate": 100,
|
||||
"no_route_fragment_rate": 0,
|
||||
"route_resolution_accuracy": null,
|
||||
"no_route_precision": null,
|
||||
"false_no_route_rate": null,
|
||||
"execution_state_consistency_rate": 100,
|
||||
"executable_with_soft_assumptions_rate": 100,
|
||||
"soft_assumption_used_fragment_rate": 100,
|
||||
"clarification_precision": null,
|
||||
"clarification_recall": null,
|
||||
"false_clarification_rate": null
|
||||
},
|
||||
"budget": {
|
||||
"requests_total": 0,
|
||||
"retries_used": 0
|
||||
},
|
||||
"clarification_eval": {
|
||||
"labeled_cases": 0,
|
||||
"true_positive": 0,
|
||||
"false_positive": 0,
|
||||
"false_negative": 0
|
||||
},
|
||||
"route_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0,
|
||||
"expected_routed_cases": 0,
|
||||
"no_route_true_positive": 0,
|
||||
"no_route_false_positive": 0
|
||||
},
|
||||
"scope_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0
|
||||
},
|
||||
"execution_state_eval": {
|
||||
"checks_total": 2,
|
||||
"checks_passed": 2
|
||||
},
|
||||
"route_distribution": {
|
||||
"store_feature_risk": 1,
|
||||
"hybrid_store_plus_live": 1
|
||||
},
|
||||
"fallback_distribution": {
|
||||
"none": 2
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"case_id": "BQ-001",
|
||||
"raw_question": "Проверь счет 60 за июнь 2020",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "3a_NZFB9MhKiKT",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-002",
|
||||
"raw_question": "Покажи риски по счету 97",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "BrD8ORrgm8qsAy",
|
||||
"request_count_for_case": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
{
|
||||
"run_id": "eval-wWlVBoabZW",
|
||||
"timestamp": "2026-04-11T10:14:44.777Z",
|
||||
"mode": "single-pass-strict",
|
||||
"use_mock": true,
|
||||
"prompt_version": "normalizer_v2_0_2",
|
||||
"schema_version": "v2_0_2",
|
||||
"dataset": {
|
||||
"source": "inline_raw_questions",
|
||||
"file": null,
|
||||
"raw_questions_count": 2
|
||||
},
|
||||
"cases_total": 2,
|
||||
"metrics": {
|
||||
"schema_validation_pass_rate": 100,
|
||||
"scope_detection_accuracy": null,
|
||||
"scope_in_scope_rate": 100,
|
||||
"multi_intent_detected_rate": 0,
|
||||
"clarification_required_rate": 0,
|
||||
"avg_fragments_per_message": 1,
|
||||
"out_of_scope_fragment_rate": 0,
|
||||
"routed_fragment_rate": 100,
|
||||
"no_route_fragment_rate": 0,
|
||||
"route_resolution_accuracy": null,
|
||||
"no_route_precision": null,
|
||||
"false_no_route_rate": null,
|
||||
"execution_state_consistency_rate": 100,
|
||||
"executable_with_soft_assumptions_rate": 100,
|
||||
"soft_assumption_used_fragment_rate": 100,
|
||||
"clarification_precision": null,
|
||||
"clarification_recall": null,
|
||||
"false_clarification_rate": null
|
||||
},
|
||||
"budget": {
|
||||
"requests_total": 0,
|
||||
"retries_used": 0
|
||||
},
|
||||
"clarification_eval": {
|
||||
"labeled_cases": 0,
|
||||
"true_positive": 0,
|
||||
"false_positive": 0,
|
||||
"false_negative": 0
|
||||
},
|
||||
"route_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0,
|
||||
"expected_routed_cases": 0,
|
||||
"no_route_true_positive": 0,
|
||||
"no_route_false_positive": 0
|
||||
},
|
||||
"scope_eval": {
|
||||
"labeled_cases": 0,
|
||||
"correct_cases": 0
|
||||
},
|
||||
"execution_state_eval": {
|
||||
"checks_total": 2,
|
||||
"checks_passed": 2
|
||||
},
|
||||
"route_distribution": {
|
||||
"store_feature_risk": 1,
|
||||
"hybrid_store_plus_live": 1
|
||||
},
|
||||
"fallback_distribution": {
|
||||
"none": 2
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"case_id": "BQ-001",
|
||||
"raw_question": "Проверь счет 60 за июнь 2020",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "3CW7yHQOH32rKS",
|
||||
"request_count_for_case": 0
|
||||
},
|
||||
{
|
||||
"case_id": "BQ-002",
|
||||
"raw_question": "Покажи риски по счету 97",
|
||||
"validation_passed": true,
|
||||
"message_in_scope": true,
|
||||
"scope_confidence": "high",
|
||||
"contains_multiple_tasks": false,
|
||||
"fragments_total": 1,
|
||||
"in_scope_fragments": 1,
|
||||
"out_of_scope_fragments": 0,
|
||||
"unclear_fragments": 0,
|
||||
"fallback_type": "none",
|
||||
"predicted_route_status": "routed",
|
||||
"expected_route_status": null,
|
||||
"predicted_no_route_reason": null,
|
||||
"expected_no_route_reason": null,
|
||||
"predicted_clarification_required": false,
|
||||
"expected_clarification_required": null,
|
||||
"executable_with_soft_assumptions_fragments": 1,
|
||||
"trace_id": "Gdw6ytd-K55hUm",
|
||||
"request_count_for_case": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
Binary file not shown.
Loading…
Reference in New Issue