SILVER/NODEDC_1C
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Initial import NDC_1C

Browse Source
This commit is contained in:
dctouch
2026-03-26 10:38:25 +03:00
commit a162d77ef7
2943 changed files with 3615871 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ARCH/validation_run_2026-03-23_semantic_v2/benchmark_final_verdict.md
+19
View File
@@ -0,0 +1,19 @@
# Benchmark Final Verdict
## Verdict
`adopt_with_improvements`
## Key numbers
- Questions total: `35`
- Route mismatches: `7`
- Degraded answers: `0`
- Avg latency ms: `506.43`
- p95 latency ms: `1024.5`
## Recommendation
1. Fix ontology unknown mapping hotspots.
2. Tune heavy-route threshold (`store_feature_risk` vs `batch_refresh_then_store`).
3. Implement full production orchestration runtime.
docs/ARCH/validation_run_2026-03-23_semantic_v2/benchmark_questions_35.md
+39
View File
@@ -0,0 +1,39 @@
# Benchmark Questions (35)
| ID | Class | Expected route | Question |
| --- | --- | --- | --- |
| Q01 | simple_factual | store_canonical | Сальдо счета 68.02 за июнь 2020? |
| Q02 | simple_factual | live_mcp_drilldown | Документ по номеру и его ссылка. |
| Q03 | simple_factual | store_canonical | Типовая проводка по реализации. |
| Q04 | simple_factual | store_canonical | Контрагент с максимумом оборота. |
| Q05 | simple_factual | store_canonical | Договоры топ-контрагента. |
| Q06 | drilldown_explain | hybrid_store_plus_live | Объясни сальдо через движения. |
| Q07 | drilldown_explain | live_mcp_drilldown | Почему проводка на этот счет? |
| Q08 | drilldown_explain | live_mcp_drilldown | Цепочка документ -> проводки -> субконто. |
| Q09 | drilldown_explain | live_mcp_drilldown | Источник регистра для строки движения. |
| Q10 | drilldown_explain | live_mcp_drilldown | Почему выбрано это субконто3? |
| Q11 | cross_entity | hybrid_store_plus_live | Свяжи документы покупателей и проводки. |
| Q12 | cross_entity | hybrid_store_plus_live | Свяжи контрагентов, договоры и проводки. |
| Q13 | cross_entity | store_canonical | Номенклатура, склад, обороты за июнь. |
| Q14 | cross_entity | hybrid_store_plus_live | Регистр и первичный документ. |
| Q15 | cross_entity | store_canonical | По счету: контрагенты и договоры. |
| Q16 | period_trend | store_feature_risk | Обороты июня против мая. |
| Q17 | period_trend | store_feature_risk | Недельные всплески в июне. |
| Q18 | period_trend | store_feature_risk | Кто дал резкий рост активности. |
| Q19 | period_trend | store_feature_risk | Аномальный рост расходных операций? |
| Q20 | period_trend | store_feature_risk | Динамика НДС к соседним периодам. |
| Q21 | anomaly_control | store_feature_risk | Нетипичные корреспонденции счетов. |
| Q22 | anomaly_control | store_feature_risk | Незакрытые хвосты по расчетам. |
| Q23 | anomaly_control | store_feature_risk | Дублирующиеся проводки. |
| Q24 | anomaly_control | store_feature_risk | Пустые или странные субконто. |
| Q25 | anomaly_control | store_feature_risk | Узлы с подозрительно большим degree. |
| Q26 | heavy_analytical | batch_refresh_then_store | Полный риск-срез за июнь. |
| Q27 | heavy_analytical | batch_refresh_then_store | Рейтинг риск-счетов. |
| Q28 | heavy_analytical | batch_refresh_then_store | Рейтинг риск-контрагентов. |
| Q29 | heavy_analytical | store_feature_risk | Baseline closed/open periods. |
| Q30 | heavy_analytical | batch_refresh_then_store | Company anomaly summary. |
| Q31 | ambiguous_fuzzy | store_feature_risk | Что по налогам и рискам? |
| Q32 | ambiguous_fuzzy | store_feature_risk | Что странное в расходах? |
| Q33 | ambiguous_fuzzy | store_feature_risk | Самые рисковые контрагенты? |
| Q34 | ambiguous_fuzzy | hybrid_store_plus_live | Что с 68.02? |
| Q35 | ambiguous_fuzzy | store_feature_risk | Проверить документы июня. |
docs/ARCH/validation_run_2026-03-23_semantic_v2/benchmark_route_analysis.md
+19
View File
@@ -0,0 +1,19 @@
# Benchmark Route Analysis
- Total mismatches: `7`
## Route confusion matrix
- `batch_refresh_then_store` -> store_feature_risk:4
- `hybrid_store_plus_live` -> hybrid_store_plus_live:3, store_canonical:2
- `live_mcp_drilldown` -> hybrid_store_plus_live:1, live_mcp_drilldown:4
- `store_canonical` -> store_canonical:6
- `store_feature_risk` -> store_feature_risk:15
## Mismatch by class
| Class | Mismatch count |
| --- | --- |
| cross_entity | 2 |
| drilldown_explain | 1 |
| heavy_analytical | 4 |
docs/ARCH/validation_run_2026-03-23_semantic_v2/benchmark_run_report.md
+38
View File
@@ -0,0 +1,38 @@
# Benchmark Run Report
## Aggregate statistics
| Metric | Value |
| --- | --- |
| questions_total | 35 |
| avg_latency_ms | 506.43 |
| median_latency_ms | 402 |
| p90_latency_ms | 941.4 |
| p95_latency_ms | 1024.5 |
| avg_context_size | 2162.51 |
| live_route_count | 8 |
| store_route_count | 27 |
| batch_route_count | 0 |
| route_mismatch_count | 7 |
| degraded_answers_count | 0 |
## Route distribution
| Route | Count |
| --- | --- |
| hybrid_store_plus_live | 4 |
| live_mcp_drilldown | 4 |
| store_canonical | 8 |
| store_feature_risk | 19 |
## Question class distribution
| Class | Count |
| --- | --- |
| ambiguous_fuzzy | 5 |
| anomaly_control | 5 |
| cross_entity | 5 |
| drilldown_explain | 5 |
| heavy_analytical | 5 |
| period_trend | 5 |
| simple_factual | 5 |
docs/ARCH/validation_run_2026-03-23_semantic_v2/benchmark_run_results.json
+827
View File
@@ -0,0 +1,827 @@
{
"status": "success",
"slice_window_key": "2020-06",
"generated_at": "2026-03-23T10:22:58.317178+00:00",
"questions_total": 35,
"aggregate": {
"questions_total": 35,
"avg_latency_ms": 506.43,
"median_latency_ms": 402,
"p90_latency_ms": 941.4,
"p95_latency_ms": 1024.5,
"avg_context_size": 2162.51,
"live_route_count": 8,
"store_route_count": 27,
"batch_route_count": 0,
"route_mismatch_count": 7,
"degraded_answers_count": 0,
"route_distribution": {
"store_canonical": 8,
"live_mcp_drilldown": 4,
"hybrid_store_plus_live": 4,
"store_feature_risk": 19
},
"question_class_distribution": {
"simple_factual": 5,
"drilldown_explain": 5,
"cross_entity": 5,
"period_trend": 5,
"anomaly_control": 5,
"heavy_analytical": 5,
"ambiguous_fuzzy": 5
}
},
"results": [
{
"question_id": "Q01",
"question_text": "Сальдо счета 68.02 за июнь 2020?",
"question_class": "simple_factual",
"expected_route": "store_canonical",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 332,
"planning_time_ms": 67,
"retrieval_time_ms": 129,
"response_generation_time_ms": 136,
"context_size": 1595,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q02",
"question_text": "Документ по номеру и его ссылка.",
"question_class": "simple_factual",
"expected_route": "live_mcp_drilldown",
"actual_route": "live_mcp_drilldown",
"sources_used": [
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 1020,
"planning_time_ms": 93,
"retrieval_time_ms": 740,
"response_generation_time_ms": 187,
"context_size": 2796,
"answer_text": "[simulated-4o-mini-profile] route=live_mcp_drilldown; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q03",
"question_text": "Типовая проводка по реализации.",
"question_class": "simple_factual",
"expected_route": "store_canonical",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 338,
"planning_time_ms": 69,
"retrieval_time_ms": 131,
"response_generation_time_ms": 138,
"context_size": 1597,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q04",
"question_text": "Контрагент с максимумом оборота.",
"question_class": "simple_factual",
"expected_route": "store_canonical",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 341,
"planning_time_ms": 70,
"retrieval_time_ms": 132,
"response_generation_time_ms": 139,
"context_size": 1598,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q05",
"question_text": "Договоры топ-контрагента.",
"question_class": "simple_factual",
"expected_route": "store_canonical",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 344,
"planning_time_ms": 71,
"retrieval_time_ms": 133,
"response_generation_time_ms": 140,
"context_size": 1599,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q06",
"question_text": "Объясни сальдо через движения.",
"question_class": "drilldown_explain",
"expected_route": "hybrid_store_plus_live",
"actual_route": "hybrid_store_plus_live",
"sources_used": [
"canonical_store",
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 819,
"planning_time_ms": 114,
"retrieval_time_ms": 524,
"response_generation_time_ms": 181,
"context_size": 2950,
"answer_text": "[simulated-4o-mini-profile] route=hybrid_store_plus_live; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q07",
"question_text": "Почему проводка на этот счет?",
"question_class": "drilldown_explain",
"expected_route": "live_mcp_drilldown",
"actual_route": "live_mcp_drilldown",
"sources_used": [
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 1035,
"planning_time_ms": 98,
"retrieval_time_ms": 745,
"response_generation_time_ms": 192,
"context_size": 2801,
"answer_text": "[simulated-4o-mini-profile] route=live_mcp_drilldown; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q08",
"question_text": "Цепочка документ -> проводки -> субконто.",
"question_class": "drilldown_explain",
"expected_route": "live_mcp_drilldown",
"actual_route": "live_mcp_drilldown",
"sources_used": [
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 1038,
"planning_time_ms": 99,
"retrieval_time_ms": 746,
"response_generation_time_ms": 193,
"context_size": 2802,
"answer_text": "[simulated-4o-mini-profile] route=live_mcp_drilldown; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q09",
"question_text": "Источник регистра для строки движения.",
"question_class": "drilldown_explain",
"expected_route": "live_mcp_drilldown",
"actual_route": "hybrid_store_plus_live",
"sources_used": [
"canonical_store",
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 828,
"planning_time_ms": 117,
"retrieval_time_ms": 527,
"response_generation_time_ms": 184,
"context_size": 2953,
"answer_text": "[simulated-4o-mini-profile] route=hybrid_store_plus_live; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected live_mcp_drilldown, got hybrid_store_plus_live"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q10",
"question_text": "Почему выбрано это субконто3?",
"question_class": "drilldown_explain",
"expected_route": "live_mcp_drilldown",
"actual_route": "live_mcp_drilldown",
"sources_used": [
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 1017,
"planning_time_ms": 92,
"retrieval_time_ms": 739,
"response_generation_time_ms": 186,
"context_size": 2795,
"answer_text": "[simulated-4o-mini-profile] route=live_mcp_drilldown; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q11",
"question_text": "Свяжи документы покупателей и проводки.",
"question_class": "cross_entity",
"expected_route": "hybrid_store_plus_live",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 335,
"planning_time_ms": 68,
"retrieval_time_ms": 130,
"response_generation_time_ms": 137,
"context_size": 1596,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected hybrid_store_plus_live, got store_canonical"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q12",
"question_text": "Свяжи контрагентов, договоры и проводки.",
"question_class": "cross_entity",
"expected_route": "hybrid_store_plus_live",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 338,
"planning_time_ms": 69,
"retrieval_time_ms": 131,
"response_generation_time_ms": 138,
"context_size": 1597,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected hybrid_store_plus_live, got store_canonical"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q13",
"question_text": "Номенклатура, склад, обороты за июнь.",
"question_class": "cross_entity",
"expected_route": "store_canonical",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 341,
"planning_time_ms": 70,
"retrieval_time_ms": 132,
"response_generation_time_ms": 139,
"context_size": 1598,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q14",
"question_text": "Регистр и первичный документ.",
"question_class": "cross_entity",
"expected_route": "hybrid_store_plus_live",
"actual_route": "hybrid_store_plus_live",
"sources_used": [
"canonical_store",
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 816,
"planning_time_ms": 113,
"retrieval_time_ms": 523,
"response_generation_time_ms": 180,
"context_size": 2949,
"answer_text": "[simulated-4o-mini-profile] route=hybrid_store_plus_live; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q15",
"question_text": "По счету: контрагенты и договоры.",
"question_class": "cross_entity",
"expected_route": "store_canonical",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 347,
"planning_time_ms": 72,
"retrieval_time_ms": 134,
"response_generation_time_ms": 141,
"context_size": 1600,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q16",
"question_text": "Обороты июня против мая.",
"question_class": "period_trend",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 402,
"planning_time_ms": 85,
"retrieval_time_ms": 155,
"response_generation_time_ms": 162,
"context_size": 2101,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q17",
"question_text": "Недельные всплески в июне.",
"question_class": "period_trend",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 405,
"planning_time_ms": 86,
"retrieval_time_ms": 156,
"response_generation_time_ms": 163,
"context_size": 2102,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q18",
"question_text": "Кто дал резкий рост активности.",
"question_class": "period_trend",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 408,
"planning_time_ms": 87,
"retrieval_time_ms": 157,
"response_generation_time_ms": 164,
"context_size": 2103,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q19",
"question_text": "Аномальный рост расходных операций?",
"question_class": "period_trend",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 411,
"planning_time_ms": 88,
"retrieval_time_ms": 158,
"response_generation_time_ms": 165,
"context_size": 2104,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q20",
"question_text": "Динамика НДС к соседним периодам.",
"question_class": "period_trend",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 387,
"planning_time_ms": 80,
"retrieval_time_ms": 150,
"response_generation_time_ms": 157,
"context_size": 2096,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q21",
"question_text": "Нетипичные корреспонденции счетов.",
"question_class": "anomaly_control",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 390,
"planning_time_ms": 81,
"retrieval_time_ms": 151,
"response_generation_time_ms": 158,
"context_size": 2097,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q22",
"question_text": "Незакрытые хвосты по расчетам.",
"question_class": "anomaly_control",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 393,
"planning_time_ms": 82,
"retrieval_time_ms": 152,
"response_generation_time_ms": 159,
"context_size": 2098,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q23",
"question_text": "Дублирующиеся проводки.",
"question_class": "anomaly_control",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 396,
"planning_time_ms": 83,
"retrieval_time_ms": 153,
"response_generation_time_ms": 160,
"context_size": 2099,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q24",
"question_text": "Пустые или странные субконто.",
"question_class": "anomaly_control",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 399,
"planning_time_ms": 84,
"retrieval_time_ms": 154,
"response_generation_time_ms": 161,
"context_size": 2100,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q25",
"question_text": "Узлы с подозрительно большим degree.",
"question_class": "anomaly_control",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 402,
"planning_time_ms": 85,
"retrieval_time_ms": 155,
"response_generation_time_ms": 162,
"context_size": 2101,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q26",
"question_text": "Полный риск-срез за июнь.",
"question_class": "heavy_analytical",
"expected_route": "batch_refresh_then_store",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 405,
"planning_time_ms": 86,
"retrieval_time_ms": 156,
"response_generation_time_ms": 163,
"context_size": 2102,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected batch_refresh_then_store, got store_feature_risk"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q27",
"question_text": "Рейтинг риск-счетов.",
"question_class": "heavy_analytical",
"expected_route": "batch_refresh_then_store",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 408,
"planning_time_ms": 87,
"retrieval_time_ms": 157,
"response_generation_time_ms": 164,
"context_size": 2103,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected batch_refresh_then_store, got store_feature_risk"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q28",
"question_text": "Рейтинг риск-контрагентов.",
"question_class": "heavy_analytical",
"expected_route": "batch_refresh_then_store",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 411,
"planning_time_ms": 88,
"retrieval_time_ms": 158,
"response_generation_time_ms": 165,
"context_size": 2104,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected batch_refresh_then_store, got store_feature_risk"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q29",
"question_text": "Baseline closed/open periods.",
"question_class": "heavy_analytical",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 414,
"planning_time_ms": 89,
"retrieval_time_ms": 159,
"response_generation_time_ms": 166,
"context_size": 2105,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q30",
"question_text": "Company anomaly summary.",
"question_class": "heavy_analytical",
"expected_route": "batch_refresh_then_store",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 390,
"planning_time_ms": 81,
"retrieval_time_ms": 151,
"response_generation_time_ms": 158,
"context_size": 2097,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected batch_refresh_then_store, got store_feature_risk"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q31",
"question_text": "Что по налогам и рискам?",
"question_class": "ambiguous_fuzzy",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 393,
"planning_time_ms": 82,
"retrieval_time_ms": 152,
"response_generation_time_ms": 159,
"context_size": 2098,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q32",
"question_text": "Что странное в расходах?",
"question_class": "ambiguous_fuzzy",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 396,
"planning_time_ms": 83,
"retrieval_time_ms": 153,
"response_generation_time_ms": 160,
"context_size": 2099,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q33",
"question_text": "Самые рисковые контрагенты?",
"question_class": "ambiguous_fuzzy",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 399,
"planning_time_ms": 84,
"retrieval_time_ms": 154,
"response_generation_time_ms": 161,
"context_size": 2100,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q34",
"question_text": "Что с 68.02?",
"question_class": "ambiguous_fuzzy",
"expected_route": "hybrid_store_plus_live",
"actual_route": "hybrid_store_plus_live",
"sources_used": [
"canonical_store",
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 822,
"planning_time_ms": 115,
"retrieval_time_ms": 525,
"response_generation_time_ms": 182,
"context_size": 2951,
"answer_text": "[simulated-4o-mini-profile] route=hybrid_store_plus_live; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q35",
"question_text": "Проверить документы июня.",
"question_class": "ambiguous_fuzzy",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 405,
"planning_time_ms": 86,
"retrieval_time_ms": 156,
"response_generation_time_ms": 163,
"context_size": 2102,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
}
]
}
docs/ARCH/validation_run_2026-03-23_semantic_v2/check_validation_run_accounting_analytics_result_2026-03-23.md
+138
View File
@@ -0,0 +1,138 @@
# CHECK Validation Run Accounting Analytics — Result (2026-03-23)
## Scope
Выполнен ремонт ontology/mapping слоя по чек-листу `IN/CHEK_Validation_Run_Accounting_Analytics.md`:
1. Расширены canonical entity classes под недостающие доменные роли.
2. Переписан mapper на semantic-v2 подход:
- entity type resolver (с приоритетом `*_Type`);
- relation resolver (role/context-aware relations вместо одного `reference`);
- null GUID filter;
- composite source id builder.
3. Проведён re-map июньского snapshot 2020 и повторный validation-run.
## Implemented changes
### 1) Canonical model expansion
Добавлены классы:
- `ResponsiblePerson`
- `Currency`
- `Warehouse`
- `CashflowArticle`
- `Department`
- `Individual`
- `Item`
- `BankAccount`
- `InvoiceDocument`
- `RegisterRecord`
Файл: `canonical_layer/models.py`
### 2) Mapper architecture refactor
Файл: `canonical_layer/mappers.py`
Ключевые изменения:
- `map_record` больше не оставляет пустые/unknown `source_id`:
- сначала пробует `Ref_Key/Ref/ID/...`,
- затем строит `cmp:<sha1>` составной id по стабильному payload.
- links строятся через semantic resolvers:
- `register_recorded_by_document`
- `journal_refers_to_document`
- `document_has_counterparty`
- `document_has_contract`
- `document_belongs_to_organization`
- `document_has_currency`
- `document_has_responsible`
- `register_relates_to_supplier`
- `register_relates_to_buyer`
- `register_relates_to_invoice`
- и др.
- `*_Type` используется как приоритетный target type hint.
- `00000000-0000-0000-0000-000000000000` фильтруется из canonical links.
- Добавлен `canonical_relation_rule_catalog()` для прозрачной выгрузки правил.
### 3) Regression tests
Файл: `tests/test_mappers.py` (переписан)
Проверки включают:
- document counterparty relation;
- register composite source id;
- journal `Ref` -> document relation;
- supplier/buyer role typing;
- `СчетФактура` -> `InvoiceDocument` (без ложного `Account`);
- zero-GUID filtering.
Статус тестов:
- `python -m pytest -q` -> `17 passed`
## Re-map and validation results
### Re-map command
`python scripts/remap_snapshot_semantic_v2.py`
Артефакты:
- `logs/pre_report_snapshot_2020_2020-06_semantic_v2.json`
- `logs/pre_report_snapshot_2020_2020-06_semantic_v2_metrics.json`
### Before vs after (snapshot-level metrics)
- `source_id_unknown`: `358 -> 0`
- `unknown_links`: `1016 -> 102`
- `semantic_coverage_pct`: `61.1917 -> 94.9279`
- `relation_types_total`: `1 -> 25`
### Validation run on semantic-v2 snapshot
Команда:
`python scripts/run_validation_accounting_analytics.py --snapshot-path logs/pre_report_snapshot_2020_2020-06_semantic_v2.json --output-dir docs/ARCH/validation_run_2026-03-23_semantic_v2 --strict`
Папка результатов:
- `docs/ARCH/validation_run_2026-03-23_semantic_v2/`
Ключевые ontology-аудит метрики:
- `covered_entity_classes`: `33 -> 42`
- `uncovered_entity_classes`: `9 -> 0`
- `unknown_relations`: `1016 -> 102`
- `semantic_coverage_pct`: `61.1917 -> 94.9279`
- `relation_types_total`: `1 -> 25`
## Export package refresh
Обновлён пакет `docs/ARCH/2020экспорт` на базе semantic-v2 snapshot:
- `01_ontology_mapping_layer.md` (обновлённые классы/метрики)
- `02_canonical_relation_rules.md` (каталог semantic relations из маппера)
- все sample/json файлы перегенерированы
Команда:
`python scripts/export_arch_2020_package.py`
## Remaining gap (post-fix)
Топ остаточных unknown-полей после semantic-v2:
- `ВидОперации`
- `Информация`
- `СубконтоДт1`
- `Руководитель_Key`
- `ГлавныйБухгалтер_Key`
Следующий шаг:
- добавить точечные semantic rules для перечисленных полей;
- отдельно закрыть `Subconto*` mapping в детальный typed-slot слой;
- после этого повторить remap + validation и зафиксировать delta.
docs/ARCH/validation_run_2026-03-23_semantic_v2/llm_like_simulation_profile.md
+27
View File
@@ -0,0 +1,27 @@
# LLM-like Simulation Profile
Simulation mode: `4o-mini-like` (controlled emulation)
## Constraints
- Store-first retrieval policy.
- Compact planning and bounded context.
- Limited live calls for drill-down only.
- Avoid expensive heavy live scans.
## Route timing baseline (ms)
| Route | Planning | Retrieval | Generation | Context |
| --- | --- | --- | --- | --- |
| live_mcp_drilldown | 95 | 780 | 180 | 2900 |
| store_canonical | 70 | 170 | 130 | 1700 |
| store_feature_risk | 82 | 190 | 150 | 2200 |
| hybrid_store_plus_live | 112 | 560 | 170 | 3050 |
| batch_refresh_then_store | 135 | 1240 | 210 | 3600 |
## Active run context
- Slice window: `2020-06`
- Refresh latest run: `368e424624bb4e1d818091e6189ab222`
- Feature latest run: `d61114c3a5c5405f89ed2952a1755516`
- Risk latest run: `2785fa2100e84935a210687af6bb850b`
docs/ARCH/validation_run_2026-03-23_semantic_v2/ontology_mapping_audit.md
+50
View File
@@ -0,0 +1,50 @@
# Ontology & Mapping Audit
## Core metrics
| Metric | Value |
| --- | --- |
| entity_classes_total | 42 |
| covered_entity_classes | 42 |
| uncovered_entity_classes | 0 |
| relation_types_total | 25 |
| correctly_typed_relations | 1909 |
| unknown_relations | 102 |
| conflicting_mappings_count | 1 |
| link_coverage_pct | 100.0 |
| semantic_coverage_pct | 94.9279 |
## Top problematic source entity types
| Source entity | Unknown relations |
| --- | --- |
| DocumentJournal_БанковскиеВыписки | 30 |
| DocumentJournal_ЖурналОпераций | 16 |
| Document_СписаниеСРасчетногоСчета | 14 |
| Document_РеализацияТоваровУслуг | 12 |
| Document_СчетФактураВыданный | 8 |
| Document_ОперацияБух | 5 |
| AccumulationRegister_СтраховыеВзносыСведенияОДоходах_RecordType | 4 |
| DocumentJournal_КассовыеДокументы | 4 |
| Document_РасходныйКассовыйОрдер | 4 |
| AccumulationRegister_НДФЛСведенияОДоходах_RecordType | 3 |
| AccumulationRegister_НДФЛПредоставленныеСтандартныеВычетыФизЛиц_RecordType | 1 |
| Document_СчетНаОплатуПокупателю | 1 |
## Top problematic relation fields
| Source field | Unknown relations |
| --- | --- |
| ВидОперации | 34 |
| Информация | 16 |
| СубконтоДт1 | 15 |
| Руководитель_Key | 8 |
| ГлавныйБухгалтер_Key | 8 |
| СпособЗаполнения | 5 |
| ВидДохода_Key | 4 |
| СтатьяДоходовИРасходовПоТаре_Key | 4 |
| КодДохода_Key | 3 |
| СубконтоДт2 | 3 |
| КодВычета_Key | 1 |
| СтруктурнаяЕдиница_Key | 1 |
docs/ARCH/validation_run_2026-03-23_semantic_v2/orchestration_policy_spec.md
+37
View File
@@ -0,0 +1,37 @@
# Orchestration Policy Spec
## Decision tree
- exact object trace or posting chain -> `live_mcp_drilldown`
- simple factual in loaded slice -> `store_canonical`
- trend/anomaly/risk -> `store_feature_risk`
- heavy whole-slice with freshness gap -> `batch_refresh_then_store`
- low confidence fallback -> `hybrid_store_plus_live`
## Routing rules
- Prefer store answers when freshness allows.
- Use live bridge only for drill-down evidence.
- Do not run uncapped heavy live scans.
- Trigger refresh/features/risk for stale context.
- Apply retrieval/context budget before fallback.
## Source priorities
| Scenario | Priority order |
| --- | --- |
| simple_factual | canonical_store -> mcp_runtime_bridge |
| drilldown_explain | mcp_runtime_bridge -> canonical_store |
| period_trend | feature_store -> risk_store -> canonical_store |
| anomaly_control | risk_store -> feature_store -> canonical_store |
| heavy_analytical | batch_refresh_then_store -> feature_store -> risk_store |
| ambiguous_fuzzy | feature_store -> canonical_store -> mcp_runtime_bridge |
## Timeout budget (ms)
| Budget | Value |
| --- | --- |
| planning | 200 |
| retrieval_soft_limit | 1200 |
| retrieval_hard_limit | 2500 |
| response_generation | 600 |
docs/ARCH/validation_run_2026-03-23_semantic_v2/slice_ingestion_report.md
+20
View File
@@ -0,0 +1,20 @@
# Slice Ingestion Report
Validation date: 2026-03-23T10:22:58.316163+00:00
Slice window: `2020-06` (`2020-06-01T00:00:00+00:00` -> `2020-07-01T00:00:00+00:00`)
- Snapshot file: `logs\pre_report_snapshot_2020_2020-06_semantic_v2.json`
- Profile file: `X:\1C\NDC_1C\logs\pre_report_activity_2020.json`
- Snapshot entities: `409`
- Snapshot links: `2011`
- Refresh run id: `368e424624bb4e1d818091e6189ab222`
- Entities written: `409`
- Links written: `2011`
- Checkpoints updated: `42`
- Canonical entities total: `769`
- Canonical links total: `3700`
- Feature run status: `success`
- Feature metrics written: `202`
- Risk run status: `success`
- Risk patterns written: `3`
- Risk global score: `0.988676`