400 lines
24 KiB
JSON
400 lines
24 KiB
JSON
{
|
||
"trace_id": "fzUNOFmwmU2v_A",
|
||
"timestamp": "2026-03-23T16:26:48.588Z",
|
||
"model": "gpt-4o-mini",
|
||
"prompt_version": "normalizer_v1_1_2_1",
|
||
"schema_version": "v1",
|
||
"case_id": "V1121-B1-02",
|
||
"user_question_raw": "Где по покупателям у нас висит история \"отгрузили - денег нет - закрытия нет\", и по каким контрагентам это уже требует ручной проверки?",
|
||
"context": {
|
||
"period_hint": null,
|
||
"business_context": null,
|
||
"expected_route": "hybrid_store_plus_live",
|
||
"case_id": "V1121-B1-02",
|
||
"eval_mode": "single-pass-strict"
|
||
},
|
||
"request_payload_redacted": {
|
||
"model": "gpt-4o-mini",
|
||
"baseUrl": "https://api.openai.com/v1",
|
||
"temperature": 0,
|
||
"maxOutputTokens": 900,
|
||
"promptVersion": "normalizer_v1_1_2_1",
|
||
"systemPrompt": "Ты semantic-normalizer для бухгалтерского ассистента NDC.\nТвоя роль: только нормализация запроса пользователя в строгий JSON-контракт.\n\nЖесткие правила:\n1) Не давай бухгалтерский ответ по сути вопроса.\n2) Возвращай только JSON без markdown и пояснений.\n3) JSON обязан соответствовать переданной schema normalized_query_v1.\n4) Если период не указан, не выдумывай его; отмечай ambiguity.\n5) Для цепочек документов/проводок/оплат поднимай causal и cross-entity признаки.\n6) Для точечного object trace (номер/строка/ref) поднимай needs_exact_object_trace=true.\n7) Используй терминологию NDC.",
|
||
"developerPrompt": "You only perform normalization into JSON schema `normalized_query_v1`.\nDo not provide a final accounting answer. Return classification and routing signals only.\n\nTaxonomy intent_class:\n- cross_entity: use when the question asks to connect documents/payments/postings/contracts/registers/evidence chains across entities.\n- drilldown_explain: use for one concrete object (document number, ref, exact posting line, exact operation card).\n- rule_based_account_control: use for account-rule checks without multi-entity chain analysis.\n- anomaly_probe: use for suspicious pattern/risk-zone detection without mandatory company-wide batch analytics.\n- heavy_analytical: use for overview, ranking, concentration analysis, summary, prioritized analytical review.\n- period_close_risk: use when the core meaning is close-process failure risk (pre-close destabilization, last-day close failure).\n- ambiguous_human_query: fallback only when question truly cannot be mapped to a concrete class.\n- simple_factual: basic factual request without analytical depth.\n\nRoute_hint policy:\n- live_mcp_drilldown: exact object trace with needs_runtime_truth=true.\n- hybrid_store_plus_live: cross-entity causal chain with evidence.\n- batch_refresh_then_store: heavy analytical ranking/overview at company or broad period scale.\n- store_feature_risk: anomaly/risk/control without mandatory heavy batch aggregation.\n- store_canonical: simple factual request.\n\nRequires policy:\n- needs_cross_entity_join=true only for multi-entity joins.\n- needs_causal_chain=true only for explicit \"why/prove/chain breakdown\" semantics.\n- needs_exact_object_trace=true for concrete object id/number/ref/line.\n- needs_period_cut=true when period slice is required.\n- needs_evidence=true when documentary evidence is requested.\n\nBoundary rule: period_close_risk vs heavy_analytical (v1.1.2)\nIf a question is explicitly about period close risk, pre-close danger, last-day closing failure, reporting deadline risk, or threats that can break the close process itself, use `period_close_risk` as the primary intent_class.\n\nHowever, if the main purpose of the question is:\n- ranking,\n- top issues,\n- overview,\n- concentration of errors,\n- summary,\n- prioritized review list,\n- company-wide analytical review,\n\nthen the primary intent_class should remain `heavy_analytical`, even if the question is phrased in the context of month-end close or reporting preparation.\n\n`heavy_analytical` has priority over `period_close_risk` when the question asks for:\n- ranking,\n- top-N,\n- overview,\n- summary,\n- company-wide picture,\n- prioritized analytical review,\neven if the wording mentions closing period, reporting, or pre-close context.\n\nUse `period_close_risk` only when the core of the question is the risk of failing or destabilizing the close process itself.\n\nExact drilldown guard:\nIf the question is about one specific document/operation/ref and asks to show a related posting/source, keep:\n- needs_exact_object_trace=true\n- needs_runtime_truth=true\n- needs_cross_entity_join=false\nunless the user explicitly asks to analyze multiple entities/cases.\n\nAnomaly route guard:\nIf the question asks for suspicious zones/risk cases without ranking and without company-wide aggregation, keep route_hint=`store_feature_risk`.\n\nConfidence guard:\nIf a question is plausibly on the boundary between `heavy_analytical` and `period_close_risk`, do not assign `confidence.overall = high`.\nUse `medium` unless wording strongly and unambiguously centers on close-process failure risk rather than analytical overview.",
|
||
"domainPrompt": "Контекст домена: бухгалтерия 1С/NDC.\n\nКлючевые счета:\n- 01, 02, 10, 41, 51, 60, 62, 68.02, 90, 97.\n\nТиповые сущности:\n- контрагент, договор, документ реализации, документ поступления, оплата, проводка, регистр, закрывающий документ.\n\nЛексика causal и сверки (сильные сигналы для cross_entity):\n- \"не бьется\", \"не сходится\", \"не видно\", \"не собралось\", \"повисло\", \"хвост\";\n- \"разложи по документам/оплатам/закрывающим\";\n- \"чем подтверждается\", \"где ошибка в цепочке\", \"что пошло криво\".\n\nЛексика точечного drilldown:\n- \"документ №...\", \"ref\", \"строка проводки\", \"покажи конкретную операцию\", \"точный source-of-record\".\n\nЛексика rule-based контроля:\n- \"проверь настройки\", \"ошибка срока/даты\", \"контроль 97/10/ОС\", \"нарушение правила учета\".\n\nЛексика обзорной аналитики:\n- \"рейтинг\", \"топ рисков\", \"в целом по компании\", \"перед закрытием периода\", \"приоритизация проверок\".\n\nВажное правило:\nЕсли в одном вопросе есть и риск-лексика, и цепочка document/payment/posting, не понижать задачу до чистого `store_feature_risk`.\nПриоритет у causal cross-entity семантики.",
|
||
"fewShotExamples": "Q: По каким поставщикам на конец июня не сходится 60 счет: разложи по документам, оплатам и закрывающим.\nExpected:\n{\n \"intent_class\": \"cross_entity\",\n \"requires\": {\n \"needs_cross_entity_join\": true,\n \"needs_causal_chain\": true,\n \"needs_exact_object_trace\": false\n },\n \"expected_output_shape\": \"reconciliation_report\",\n \"route_hint\": \"hybrid_store_plus_live\"\n}\n\nQ: По 10 счету где подозрительные остатки и почему они повисли по цепочке документов?\nExpected:\n{\n \"intent_class\": \"cross_entity\",\n \"requires\": {\n \"needs_cross_entity_join\": true,\n \"needs_causal_chain\": true,\n \"needs_anomaly_summary\": true\n },\n \"expected_output_shape\": \"reconciliation_report\",\n \"route_hint\": \"hybrid_store_plus_live\"\n}\n\nQ: По 97 счету проверь ошибки дат начала/окончания списания, без разложения по документам.\nExpected:\n{\n \"intent_class\": \"rule_based_account_control\",\n \"requires\": {\n \"needs_cross_entity_join\": false,\n \"needs_causal_chain\": false\n },\n \"expected_output_shape\": \"prioritized_review_list\",\n \"route_hint\": \"store_feature_risk\"\n}\n\nQ: Покажи документ TRX-88 и связанную проводку по 51, нужен точный объект.\nExpected:\n{\n \"intent_class\": \"drilldown_explain\",\n \"requires\": {\n \"needs_exact_object_trace\": true,\n \"needs_runtime_truth\": true,\n \"needs_cross_entity_join\": false\n },\n \"expected_output_shape\": \"evidence_chain\",\n \"route_hint\": \"live_mcp_drilldown\"\n}\n\nQ: Где у нас по июню не собралось и что пошло криво в взаиморасчетах с покупателями?\nExpected:\n{\n \"intent_class\": \"cross_entity\",\n \"requires\": {\n \"needs_cross_entity_join\": true,\n \"needs_causal_chain\": true,\n \"needs_period_cut\": true\n },\n \"expected_output_shape\": \"reconciliation_report\",\n \"route_hint\": \"hybrid_store_plus_live\"\n}\n\nQ: Где у нас пахнет ручной ошибкой по июню, без конкретного документа.\nExpected:\n{\n \"intent_class\": \"ambiguous_human_query\",\n \"requires\": {\n \"needs_cross_entity_join\": false,\n \"needs_causal_chain\": false,\n \"needs_exact_object_trace\": false\n },\n \"expected_output_shape\": \"anomaly_summary\",\n \"route_hint\": \"batch_refresh_then_store\"\n}\n\n# keep period_close_risk anchor\n\nQ: Перед закрытием периода что у нас может взорваться в последний день?\nExpected:\n{\n \"intent_class\": \"period_close_risk\",\n \"requires\": {\n \"needs_period_cut\": true,\n \"needs_ranking\": false,\n \"needs_exact_object_trace\": false\n },\n \"expected_output_shape\": \"prioritized_review_list\",\n \"route_hint\": \"batch_refresh_then_store\"\n}\n\nQ: Где по июню выглядит подозрительно, просто дай зоны риска без детального разбора.\nExpected:\n{\n \"intent_class\": \"anomaly_probe\",\n \"requires\": {\n \"needs_anomaly_summary\": true,\n \"needs_period_cut\": true,\n \"needs_ranking\": false\n },\n \"expected_output_shape\": \"anomaly_summary\",\n \"route_hint\": \"store_feature_risk\"\n}\n\n# v1.1.2 boundary balancing additions (exactly 2 new heavy counterexamples)\n\nQ: Сделай рейтинг самых рисковых хвостов перед закрытием периода за июнь.\nExpected:\n{\n \"intent_class\": \"heavy_analytical\",\n \"requires\": {\n \"needs_cross_entity_join\": false,\n \"needs_causal_chain\": false,\n \"needs_exact_object_trace\": false,\n \"needs_ranking\": true,\n \"needs_anomaly_summary\": false,\n \"needs_runtime_truth\": false,\n \"needs_period_cut\": true,\n \"needs_evidence\": false\n },\n \"expected_output_shape\": \"ranked_list\",\n \"route_hint\": \"batch_refresh_then_store\"\n}\n\nQ: Дай обзорный риск-срез перед сдачей отчетности: где максимальная концентрация ошибок.\nExpected:\n{\n \"intent_class\": \"heavy_analytical\",\n \"requires\": {\n \"needs_cross_entity_join\": false,\n \"needs_causal_chain\": false,\n \"needs_exact_object_trace\": false,\n \"needs_ranking\": true,\n \"needs_anomaly_summary\": true,\n \"needs_runtime_truth\": false,\n \"needs_period_cut\": true,\n \"needs_evidence\": false\n },\n \"expected_output_shape\": \"anomaly_summary\",\n \"route_hint\": \"batch_refresh_then_store\"\n}",
|
||
"userQuestion": "Где по покупателям у нас висит история \"отгрузили - денег нет - закрытия нет\", и по каким контрагентам это уже требует ручной проверки?",
|
||
"context": {
|
||
"expected_route": "hybrid_store_plus_live",
|
||
"eval_label": "eval-h3k8TyTFuu",
|
||
"case_id": "V1121-B1-02",
|
||
"eval_mode": "single-pass-strict"
|
||
},
|
||
"retryPolicy": "single-pass-strict",
|
||
"useMock": false
|
||
},
|
||
"raw_model_response": {
|
||
"id": "resp_0e41dff99a4bfdee0069c169bf42288197ad19b54f2672715d",
|
||
"object": "response",
|
||
"created_at": 1774283199,
|
||
"status": "completed",
|
||
"background": false,
|
||
"billing": {
|
||
"payer": "developer"
|
||
},
|
||
"completed_at": 1774283208,
|
||
"error": null,
|
||
"frequency_penalty": 0,
|
||
"incomplete_details": null,
|
||
"instructions": null,
|
||
"max_output_tokens": 900,
|
||
"max_tool_calls": null,
|
||
"model": "gpt-4o-mini-2024-07-18",
|
||
"output": [
|
||
{
|
||
"id": "msg_0e41dff99a4bfdee0069c169c0a1848197b32d6d25fafd4845",
|
||
"type": "message",
|
||
"status": "completed",
|
||
"content": [
|
||
{
|
||
"type": "output_text",
|
||
"annotations": [],
|
||
"logprobs": [],
|
||
"text": "{\n \"schema_version\": \"normalized_query_v1\",\n \"user_question_raw\": \"Где по покупателям у нас висит история \\\"отгрузили - денег нет - закрытия нет\\\", и по каким контрагентам это уже требует ручной проверки?\",\n \"normalized_question\": \"Где по покупателям не сходится история отгрузок и оплат, и какие контрагенты требуют ручной проверки?\",\n \"intent_class\": \"cross_entity\",\n \"business_problem_type\": \"анализ взаиморасчетов с покупателями\",\n \"domain_entities\": [\"контрагент\", \"договор\", \"документ реализации\", \"оплата\"],\n \"accounts_mentioned\": [],\n \"documents_mentioned\": [],\n \"registers_mentioned\": [],\n \"period_scope\": {\n \"type\": \"missing\",\n \"value\": null,\n \"confidence\": \"medium\"\n },\n \"requires\": {\n \"needs_cross_entity_join\": true,\n \"needs_causal_chain\": true,\n \"needs_exact_object_trace\": false\n ,\n \"needs_ranking\": false,\n \"needs_anomaly_summary\": true,\n \"needs_runtime_truth\": false,\n \"needs_period_cut\": false,\n \"needs_evidence\": false\n },\n \"expected_output_shape\": \"reconciliation_report\",\n \"route_hint\": \"hybrid_store_plus_live\",\n \"ambiguities\": [],\n \"confidence\": {\n \"overall\": \"medium\",\n \"intent_class\": \"medium\",\n \"route_hint\": \"medium\"\n }\n}"
|
||
}
|
||
],
|
||
"role": "assistant"
|
||
}
|
||
],
|
||
"parallel_tool_calls": true,
|
||
"presence_penalty": 0,
|
||
"previous_response_id": null,
|
||
"prompt_cache_key": null,
|
||
"prompt_cache_retention": null,
|
||
"reasoning": {
|
||
"effort": null,
|
||
"summary": null
|
||
},
|
||
"safety_identifier": null,
|
||
"service_tier": "default",
|
||
"store": true,
|
||
"temperature": 0,
|
||
"text": {
|
||
"format": {
|
||
"type": "json_schema",
|
||
"description": null,
|
||
"name": "normalized_query_v1",
|
||
"schema": {
|
||
"$id": "normalized_query_v1",
|
||
"title": "Normalized Query V1",
|
||
"type": "object",
|
||
"additionalProperties": false,
|
||
"required": [
|
||
"schema_version",
|
||
"user_question_raw",
|
||
"normalized_question",
|
||
"intent_class",
|
||
"business_problem_type",
|
||
"domain_entities",
|
||
"accounts_mentioned",
|
||
"documents_mentioned",
|
||
"registers_mentioned",
|
||
"period_scope",
|
||
"requires",
|
||
"expected_output_shape",
|
||
"route_hint",
|
||
"ambiguities",
|
||
"confidence"
|
||
],
|
||
"properties": {
|
||
"schema_version": {
|
||
"type": "string",
|
||
"const": "normalized_query_v1"
|
||
},
|
||
"user_question_raw": {
|
||
"type": "string",
|
||
"minLength": 1
|
||
},
|
||
"normalized_question": {
|
||
"type": "string",
|
||
"minLength": 1
|
||
},
|
||
"intent_class": {
|
||
"type": "string",
|
||
"enum": [
|
||
"heavy_analytical",
|
||
"cross_entity",
|
||
"drilldown_explain",
|
||
"rule_based_account_control",
|
||
"anomaly_probe",
|
||
"period_close_risk",
|
||
"ambiguous_human_query",
|
||
"simple_factual"
|
||
]
|
||
},
|
||
"business_problem_type": {
|
||
"type": "string",
|
||
"minLength": 1
|
||
},
|
||
"domain_entities": {
|
||
"type": "array",
|
||
"items": {
|
||
"type": "string"
|
||
}
|
||
},
|
||
"accounts_mentioned": {
|
||
"type": "array",
|
||
"items": {
|
||
"type": "string"
|
||
}
|
||
},
|
||
"documents_mentioned": {
|
||
"type": "array",
|
||
"items": {
|
||
"type": "string"
|
||
}
|
||
},
|
||
"registers_mentioned": {
|
||
"type": "array",
|
||
"items": {
|
||
"type": "string"
|
||
}
|
||
},
|
||
"period_scope": {
|
||
"type": "object",
|
||
"additionalProperties": false,
|
||
"required": [
|
||
"type",
|
||
"value",
|
||
"confidence"
|
||
],
|
||
"properties": {
|
||
"type": {
|
||
"type": "string",
|
||
"enum": [
|
||
"explicit",
|
||
"inferred",
|
||
"missing"
|
||
]
|
||
},
|
||
"value": {
|
||
"type": [
|
||
"string",
|
||
"null"
|
||
]
|
||
},
|
||
"confidence": {
|
||
"type": "string",
|
||
"enum": [
|
||
"high",
|
||
"medium",
|
||
"low"
|
||
]
|
||
}
|
||
}
|
||
},
|
||
"requires": {
|
||
"type": "object",
|
||
"additionalProperties": false,
|
||
"required": [
|
||
"needs_cross_entity_join",
|
||
"needs_causal_chain",
|
||
"needs_exact_object_trace",
|
||
"needs_ranking",
|
||
"needs_anomaly_summary",
|
||
"needs_runtime_truth",
|
||
"needs_period_cut",
|
||
"needs_evidence"
|
||
],
|
||
"properties": {
|
||
"needs_cross_entity_join": {
|
||
"type": "boolean"
|
||
},
|
||
"needs_causal_chain": {
|
||
"type": "boolean"
|
||
},
|
||
"needs_exact_object_trace": {
|
||
"type": "boolean"
|
||
},
|
||
"needs_ranking": {
|
||
"type": "boolean"
|
||
},
|
||
"needs_anomaly_summary": {
|
||
"type": "boolean"
|
||
},
|
||
"needs_runtime_truth": {
|
||
"type": "boolean"
|
||
},
|
||
"needs_period_cut": {
|
||
"type": "boolean"
|
||
},
|
||
"needs_evidence": {
|
||
"type": "boolean"
|
||
}
|
||
}
|
||
},
|
||
"expected_output_shape": {
|
||
"type": "string",
|
||
"enum": [
|
||
"ranked_list",
|
||
"evidence_chain",
|
||
"anomaly_summary",
|
||
"point_answer",
|
||
"reconciliation_report",
|
||
"prioritized_review_list"
|
||
]
|
||
},
|
||
"route_hint": {
|
||
"type": "string",
|
||
"enum": [
|
||
"store_canonical",
|
||
"store_feature_risk",
|
||
"hybrid_store_plus_live",
|
||
"live_mcp_drilldown",
|
||
"batch_refresh_then_store"
|
||
]
|
||
},
|
||
"ambiguities": {
|
||
"type": "array",
|
||
"items": {
|
||
"type": "object",
|
||
"additionalProperties": false,
|
||
"required": [
|
||
"field",
|
||
"reason",
|
||
"severity"
|
||
],
|
||
"properties": {
|
||
"field": {
|
||
"type": "string"
|
||
},
|
||
"reason": {
|
||
"type": "string"
|
||
},
|
||
"severity": {
|
||
"type": "string",
|
||
"enum": [
|
||
"low",
|
||
"medium",
|
||
"high"
|
||
]
|
||
}
|
||
}
|
||
}
|
||
},
|
||
"confidence": {
|
||
"type": "object",
|
||
"additionalProperties": false,
|
||
"required": [
|
||
"overall",
|
||
"intent_class",
|
||
"route_hint"
|
||
],
|
||
"properties": {
|
||
"overall": {
|
||
"type": "string",
|
||
"enum": [
|
||
"high",
|
||
"medium",
|
||
"low"
|
||
]
|
||
},
|
||
"intent_class": {
|
||
"type": "string",
|
||
"enum": [
|
||
"high",
|
||
"medium",
|
||
"low"
|
||
]
|
||
},
|
||
"route_hint": {
|
||
"type": "string",
|
||
"enum": [
|
||
"high",
|
||
"medium",
|
||
"low"
|
||
]
|
||
}
|
||
}
|
||
}
|
||
}
|
||
},
|
||
"strict": true
|
||
},
|
||
"verbosity": "medium"
|
||
},
|
||
"tool_choice": "auto",
|
||
"tools": [],
|
||
"top_logprobs": 0,
|
||
"top_p": 1,
|
||
"truncation": "disabled",
|
||
"usage": {
|
||
"input_tokens": 2884,
|
||
"input_tokens_details": {
|
||
"cached_tokens": 0
|
||
},
|
||
"output_tokens": 328,
|
||
"output_tokens_details": {
|
||
"reasoning_tokens": 0
|
||
},
|
||
"total_tokens": 3212
|
||
},
|
||
"user": null,
|
||
"metadata": {}
|
||
},
|
||
"parsed_normalized_json": {
|
||
"schema_version": "normalized_query_v1",
|
||
"user_question_raw": "Где по покупателям у нас висит история \"отгрузили - денег нет - закрытия нет\", и по каким контрагентам это уже требует ручной проверки?",
|
||
"normalized_question": "Где по покупателям не сходится история отгрузок и оплат, и какие контрагенты требуют ручной проверки?",
|
||
"intent_class": "cross_entity",
|
||
"business_problem_type": "анализ взаиморасчетов с покупателями",
|
||
"domain_entities": [
|
||
"контрагент",
|
||
"договор",
|
||
"документ реализации",
|
||
"оплата"
|
||
],
|
||
"accounts_mentioned": [],
|
||
"documents_mentioned": [],
|
||
"registers_mentioned": [],
|
||
"period_scope": {
|
||
"type": "missing",
|
||
"value": null,
|
||
"confidence": "medium"
|
||
},
|
||
"requires": {
|
||
"needs_cross_entity_join": true,
|
||
"needs_causal_chain": true,
|
||
"needs_exact_object_trace": false,
|
||
"needs_ranking": false,
|
||
"needs_anomaly_summary": true,
|
||
"needs_runtime_truth": false,
|
||
"needs_period_cut": false,
|
||
"needs_evidence": false
|
||
},
|
||
"expected_output_shape": "reconciliation_report",
|
||
"route_hint": "hybrid_store_plus_live",
|
||
"ambiguities": [],
|
||
"confidence": {
|
||
"overall": "medium",
|
||
"intent_class": "medium",
|
||
"route_hint": "medium"
|
||
}
|
||
},
|
||
"validation_result": {
|
||
"passed": true,
|
||
"errors": []
|
||
},
|
||
"route_hint": "hybrid_store_plus_live",
|
||
"confidence": "medium",
|
||
"usage": {
|
||
"input_tokens": 2884,
|
||
"output_tokens": 328,
|
||
"total_tokens": 3212
|
||
},
|
||
"latency_ms": 9571,
|
||
"expected_route": "hybrid_store_plus_live",
|
||
"eval_label": "eval-h3k8TyTFuu",
|
||
"eval_mode": "single-pass-strict",
|
||
"request_count_for_case": 1
|
||
} |