SILVER/NODEDC_1C
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Initial import NDC_1C

Browse Source
This commit is contained in:
dctouch
2026-03-26 10:38:25 +03:00
commit a162d77ef7
2943 changed files with 3615871 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ARCH/validation_run_2026-03-23/benchmark_final_verdict.md
+19
View File
@@ -0,0 +1,19 @@
# Benchmark Final Verdict
## Verdict
`adopt_with_improvements`
## Key numbers
- Questions total: `35`
- Route mismatches: `7`
- Degraded answers: `0`
- Avg latency ms: `506.43`
- p95 latency ms: `1024.5`
## Recommendation
1. Fix ontology unknown mapping hotspots.
2. Tune heavy-route threshold (`store_feature_risk` vs `batch_refresh_then_store`).
3. Implement full production orchestration runtime.
docs/ARCH/validation_run_2026-03-23/benchmark_questions_35.md
+39
View File
@@ -0,0 +1,39 @@
# Benchmark Questions (35)
| ID | Class | Expected route | Question |
| --- | --- | --- | --- |
| Q01 | simple_factual | store_canonical | Сальдо счета 68.02 за июнь 2020? |
| Q02 | simple_factual | live_mcp_drilldown | Документ по номеру и его ссылка. |
| Q03 | simple_factual | store_canonical | Типовая проводка по реализации. |
| Q04 | simple_factual | store_canonical | Контрагент с максимумом оборота. |
| Q05 | simple_factual | store_canonical | Договоры топ-контрагента. |
| Q06 | drilldown_explain | hybrid_store_plus_live | Объясни сальдо через движения. |
| Q07 | drilldown_explain | live_mcp_drilldown | Почему проводка на этот счет? |
| Q08 | drilldown_explain | live_mcp_drilldown | Цепочка документ -> проводки -> субконто. |
| Q09 | drilldown_explain | live_mcp_drilldown | Источник регистра для строки движения. |
| Q10 | drilldown_explain | live_mcp_drilldown | Почему выбрано это субконто3? |
| Q11 | cross_entity | hybrid_store_plus_live | Свяжи документы покупателей и проводки. |
| Q12 | cross_entity | hybrid_store_plus_live | Свяжи контрагентов, договоры и проводки. |
| Q13 | cross_entity | store_canonical | Номенклатура, склад, обороты за июнь. |
| Q14 | cross_entity | hybrid_store_plus_live | Регистр и первичный документ. |
| Q15 | cross_entity | store_canonical | По счету: контрагенты и договоры. |
| Q16 | period_trend | store_feature_risk | Обороты июня против мая. |
| Q17 | period_trend | store_feature_risk | Недельные всплески в июне. |
| Q18 | period_trend | store_feature_risk | Кто дал резкий рост активности. |
| Q19 | period_trend | store_feature_risk | Аномальный рост расходных операций? |
| Q20 | period_trend | store_feature_risk | Динамика НДС к соседним периодам. |
| Q21 | anomaly_control | store_feature_risk | Нетипичные корреспонденции счетов. |
| Q22 | anomaly_control | store_feature_risk | Незакрытые хвосты по расчетам. |
| Q23 | anomaly_control | store_feature_risk | Дублирующиеся проводки. |
| Q24 | anomaly_control | store_feature_risk | Пустые или странные субконто. |
| Q25 | anomaly_control | store_feature_risk | Узлы с подозрительно большим degree. |
| Q26 | heavy_analytical | batch_refresh_then_store | Полный риск-срез за июнь. |
| Q27 | heavy_analytical | batch_refresh_then_store | Рейтинг риск-счетов. |
| Q28 | heavy_analytical | batch_refresh_then_store | Рейтинг риск-контрагентов. |
| Q29 | heavy_analytical | store_feature_risk | Baseline closed/open periods. |
| Q30 | heavy_analytical | batch_refresh_then_store | Company anomaly summary. |
| Q31 | ambiguous_fuzzy | store_feature_risk | Что по налогам и рискам? |
| Q32 | ambiguous_fuzzy | store_feature_risk | Что странное в расходах? |
| Q33 | ambiguous_fuzzy | store_feature_risk | Самые рисковые контрагенты? |
| Q34 | ambiguous_fuzzy | hybrid_store_plus_live | Что с 68.02? |
| Q35 | ambiguous_fuzzy | store_feature_risk | Проверить документы июня. |
docs/ARCH/validation_run_2026-03-23/benchmark_route_analysis.md
+19
View File
@@ -0,0 +1,19 @@
# Benchmark Route Analysis
- Total mismatches: `7`
## Route confusion matrix
- `batch_refresh_then_store` -> store_feature_risk:4
- `hybrid_store_plus_live` -> hybrid_store_plus_live:3, store_canonical:2
- `live_mcp_drilldown` -> hybrid_store_plus_live:1, live_mcp_drilldown:4
- `store_canonical` -> store_canonical:6
- `store_feature_risk` -> store_feature_risk:15
## Mismatch by class
| Class | Mismatch count |
| --- | --- |
| cross_entity | 2 |
| drilldown_explain | 1 |
| heavy_analytical | 4 |
docs/ARCH/validation_run_2026-03-23/benchmark_run_report.md
+38
View File
@@ -0,0 +1,38 @@
# Benchmark Run Report
## Aggregate statistics
| Metric | Value |
| --- | --- |
| questions_total | 35 |
| avg_latency_ms | 506.43 |
| median_latency_ms | 402 |
| p90_latency_ms | 941.4 |
| p95_latency_ms | 1024.5 |
| avg_context_size | 2162.51 |
| live_route_count | 8 |
| store_route_count | 27 |
| batch_route_count | 0 |
| route_mismatch_count | 7 |
| degraded_answers_count | 0 |
## Route distribution
| Route | Count |
| --- | --- |
| hybrid_store_plus_live | 4 |
| live_mcp_drilldown | 4 |
| store_canonical | 8 |
| store_feature_risk | 19 |
## Question class distribution
| Class | Count |
| --- | --- |
| ambiguous_fuzzy | 5 |
| anomaly_control | 5 |
| cross_entity | 5 |
| drilldown_explain | 5 |
| heavy_analytical | 5 |
| period_trend | 5 |
| simple_factual | 5 |
docs/ARCH/validation_run_2026-03-23/benchmark_run_results.json
+827
View File
@@ -0,0 +1,827 @@
{
"status": "success",
"slice_window_key": "2020-06",
"generated_at": "2026-03-23T09:28:12.312411+00:00",
"questions_total": 35,
"aggregate": {
"questions_total": 35,
"avg_latency_ms": 506.43,
"median_latency_ms": 402,
"p90_latency_ms": 941.4,
"p95_latency_ms": 1024.5,
"avg_context_size": 2162.51,
"live_route_count": 8,
"store_route_count": 27,
"batch_route_count": 0,
"route_mismatch_count": 7,
"degraded_answers_count": 0,
"route_distribution": {
"store_canonical": 8,
"live_mcp_drilldown": 4,
"hybrid_store_plus_live": 4,
"store_feature_risk": 19
},
"question_class_distribution": {
"simple_factual": 5,
"drilldown_explain": 5,
"cross_entity": 5,
"period_trend": 5,
"anomaly_control": 5,
"heavy_analytical": 5,
"ambiguous_fuzzy": 5
}
},
"results": [
{
"question_id": "Q01",
"question_text": "Сальдо счета 68.02 за июнь 2020?",
"question_class": "simple_factual",
"expected_route": "store_canonical",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 332,
"planning_time_ms": 67,
"retrieval_time_ms": 129,
"response_generation_time_ms": 136,
"context_size": 1595,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q02",
"question_text": "Документ по номеру и его ссылка.",
"question_class": "simple_factual",
"expected_route": "live_mcp_drilldown",
"actual_route": "live_mcp_drilldown",
"sources_used": [
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 1020,
"planning_time_ms": 93,
"retrieval_time_ms": 740,
"response_generation_time_ms": 187,
"context_size": 2796,
"answer_text": "[simulated-4o-mini-profile] route=live_mcp_drilldown; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q03",
"question_text": "Типовая проводка по реализации.",
"question_class": "simple_factual",
"expected_route": "store_canonical",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 338,
"planning_time_ms": 69,
"retrieval_time_ms": 131,
"response_generation_time_ms": 138,
"context_size": 1597,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q04",
"question_text": "Контрагент с максимумом оборота.",
"question_class": "simple_factual",
"expected_route": "store_canonical",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 341,
"planning_time_ms": 70,
"retrieval_time_ms": 132,
"response_generation_time_ms": 139,
"context_size": 1598,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q05",
"question_text": "Договоры топ-контрагента.",
"question_class": "simple_factual",
"expected_route": "store_canonical",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 344,
"planning_time_ms": 71,
"retrieval_time_ms": 133,
"response_generation_time_ms": 140,
"context_size": 1599,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q06",
"question_text": "Объясни сальдо через движения.",
"question_class": "drilldown_explain",
"expected_route": "hybrid_store_plus_live",
"actual_route": "hybrid_store_plus_live",
"sources_used": [
"canonical_store",
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 819,
"planning_time_ms": 114,
"retrieval_time_ms": 524,
"response_generation_time_ms": 181,
"context_size": 2950,
"answer_text": "[simulated-4o-mini-profile] route=hybrid_store_plus_live; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q07",
"question_text": "Почему проводка на этот счет?",
"question_class": "drilldown_explain",
"expected_route": "live_mcp_drilldown",
"actual_route": "live_mcp_drilldown",
"sources_used": [
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 1035,
"planning_time_ms": 98,
"retrieval_time_ms": 745,
"response_generation_time_ms": 192,
"context_size": 2801,
"answer_text": "[simulated-4o-mini-profile] route=live_mcp_drilldown; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q08",
"question_text": "Цепочка документ -> проводки -> субконто.",
"question_class": "drilldown_explain",
"expected_route": "live_mcp_drilldown",
"actual_route": "live_mcp_drilldown",
"sources_used": [
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 1038,
"planning_time_ms": 99,
"retrieval_time_ms": 746,
"response_generation_time_ms": 193,
"context_size": 2802,
"answer_text": "[simulated-4o-mini-profile] route=live_mcp_drilldown; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q09",
"question_text": "Источник регистра для строки движения.",
"question_class": "drilldown_explain",
"expected_route": "live_mcp_drilldown",
"actual_route": "hybrid_store_plus_live",
"sources_used": [
"canonical_store",
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 828,
"planning_time_ms": 117,
"retrieval_time_ms": 527,
"response_generation_time_ms": 184,
"context_size": 2953,
"answer_text": "[simulated-4o-mini-profile] route=hybrid_store_plus_live; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected live_mcp_drilldown, got hybrid_store_plus_live"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q10",
"question_text": "Почему выбрано это субконто3?",
"question_class": "drilldown_explain",
"expected_route": "live_mcp_drilldown",
"actual_route": "live_mcp_drilldown",
"sources_used": [
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 1017,
"planning_time_ms": 92,
"retrieval_time_ms": 739,
"response_generation_time_ms": 186,
"context_size": 2795,
"answer_text": "[simulated-4o-mini-profile] route=live_mcp_drilldown; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q11",
"question_text": "Свяжи документы покупателей и проводки.",
"question_class": "cross_entity",
"expected_route": "hybrid_store_plus_live",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 335,
"planning_time_ms": 68,
"retrieval_time_ms": 130,
"response_generation_time_ms": 137,
"context_size": 1596,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected hybrid_store_plus_live, got store_canonical"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q12",
"question_text": "Свяжи контрагентов, договоры и проводки.",
"question_class": "cross_entity",
"expected_route": "hybrid_store_plus_live",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 338,
"planning_time_ms": 69,
"retrieval_time_ms": 131,
"response_generation_time_ms": 138,
"context_size": 1597,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected hybrid_store_plus_live, got store_canonical"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q13",
"question_text": "Номенклатура, склад, обороты за июнь.",
"question_class": "cross_entity",
"expected_route": "store_canonical",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 341,
"planning_time_ms": 70,
"retrieval_time_ms": 132,
"response_generation_time_ms": 139,
"context_size": 1598,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q14",
"question_text": "Регистр и первичный документ.",
"question_class": "cross_entity",
"expected_route": "hybrid_store_plus_live",
"actual_route": "hybrid_store_plus_live",
"sources_used": [
"canonical_store",
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 816,
"planning_time_ms": 113,
"retrieval_time_ms": 523,
"response_generation_time_ms": 180,
"context_size": 2949,
"answer_text": "[simulated-4o-mini-profile] route=hybrid_store_plus_live; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q15",
"question_text": "По счету: контрагенты и договоры.",
"question_class": "cross_entity",
"expected_route": "store_canonical",
"actual_route": "store_canonical",
"sources_used": [
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 347,
"planning_time_ms": 72,
"retrieval_time_ms": 134,
"response_generation_time_ms": 141,
"context_size": 1600,
"answer_text": "[simulated-4o-mini-profile] route=store_canonical; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q16",
"question_text": "Обороты июня против мая.",
"question_class": "period_trend",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 402,
"planning_time_ms": 85,
"retrieval_time_ms": 155,
"response_generation_time_ms": 162,
"context_size": 2101,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q17",
"question_text": "Недельные всплески в июне.",
"question_class": "period_trend",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 405,
"planning_time_ms": 86,
"retrieval_time_ms": 156,
"response_generation_time_ms": 163,
"context_size": 2102,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q18",
"question_text": "Кто дал резкий рост активности.",
"question_class": "period_trend",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 408,
"planning_time_ms": 87,
"retrieval_time_ms": 157,
"response_generation_time_ms": 164,
"context_size": 2103,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q19",
"question_text": "Аномальный рост расходных операций?",
"question_class": "period_trend",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 411,
"planning_time_ms": 88,
"retrieval_time_ms": 158,
"response_generation_time_ms": 165,
"context_size": 2104,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q20",
"question_text": "Динамика НДС к соседним периодам.",
"question_class": "period_trend",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 387,
"planning_time_ms": 80,
"retrieval_time_ms": 150,
"response_generation_time_ms": 157,
"context_size": 2096,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q21",
"question_text": "Нетипичные корреспонденции счетов.",
"question_class": "anomaly_control",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 390,
"planning_time_ms": 81,
"retrieval_time_ms": 151,
"response_generation_time_ms": 158,
"context_size": 2097,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q22",
"question_text": "Незакрытые хвосты по расчетам.",
"question_class": "anomaly_control",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 393,
"planning_time_ms": 82,
"retrieval_time_ms": 152,
"response_generation_time_ms": 159,
"context_size": 2098,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q23",
"question_text": "Дублирующиеся проводки.",
"question_class": "anomaly_control",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 396,
"planning_time_ms": 83,
"retrieval_time_ms": 153,
"response_generation_time_ms": 160,
"context_size": 2099,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q24",
"question_text": "Пустые или странные субконто.",
"question_class": "anomaly_control",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 399,
"planning_time_ms": 84,
"retrieval_time_ms": 154,
"response_generation_time_ms": 161,
"context_size": 2100,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q25",
"question_text": "Узлы с подозрительно большим degree.",
"question_class": "anomaly_control",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 402,
"planning_time_ms": 85,
"retrieval_time_ms": 155,
"response_generation_time_ms": 162,
"context_size": 2101,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "good",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q26",
"question_text": "Полный риск-срез за июнь.",
"question_class": "heavy_analytical",
"expected_route": "batch_refresh_then_store",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 405,
"planning_time_ms": 86,
"retrieval_time_ms": 156,
"response_generation_time_ms": 163,
"context_size": 2102,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected batch_refresh_then_store, got store_feature_risk"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q27",
"question_text": "Рейтинг риск-счетов.",
"question_class": "heavy_analytical",
"expected_route": "batch_refresh_then_store",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 408,
"planning_time_ms": 87,
"retrieval_time_ms": 157,
"response_generation_time_ms": 164,
"context_size": 2103,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected batch_refresh_then_store, got store_feature_risk"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q28",
"question_text": "Рейтинг риск-контрагентов.",
"question_class": "heavy_analytical",
"expected_route": "batch_refresh_then_store",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 411,
"planning_time_ms": 88,
"retrieval_time_ms": 158,
"response_generation_time_ms": 165,
"context_size": 2104,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected batch_refresh_then_store, got store_feature_risk"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q29",
"question_text": "Baseline closed/open periods.",
"question_class": "heavy_analytical",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 414,
"planning_time_ms": 89,
"retrieval_time_ms": 159,
"response_generation_time_ms": 166,
"context_size": 2105,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q30",
"question_text": "Company anomaly summary.",
"question_class": "heavy_analytical",
"expected_route": "batch_refresh_then_store",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 390,
"planning_time_ms": 81,
"retrieval_time_ms": 151,
"response_generation_time_ms": 158,
"context_size": 2097,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "acceptable_with_warning",
"issues_detected": [
"Route mismatch: expected batch_refresh_then_store, got store_feature_risk"
],
"recommended_fix": "Tune router threshold for heavy/live boundary."
},
{
"question_id": "Q31",
"question_text": "Что по налогам и рискам?",
"question_class": "ambiguous_fuzzy",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 393,
"planning_time_ms": 82,
"retrieval_time_ms": 152,
"response_generation_time_ms": 159,
"context_size": 2098,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q32",
"question_text": "Что странное в расходах?",
"question_class": "ambiguous_fuzzy",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 396,
"planning_time_ms": 83,
"retrieval_time_ms": 153,
"response_generation_time_ms": 160,
"context_size": 2099,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q33",
"question_text": "Самые рисковые контрагенты?",
"question_class": "ambiguous_fuzzy",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 399,
"planning_time_ms": 84,
"retrieval_time_ms": 154,
"response_generation_time_ms": 161,
"context_size": 2100,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q34",
"question_text": "Что с 68.02?",
"question_class": "ambiguous_fuzzy",
"expected_route": "hybrid_store_plus_live",
"actual_route": "hybrid_store_plus_live",
"sources_used": [
"canonical_store",
"mcp_runtime_bridge"
],
"refresh_needed": false,
"latency_ms": 822,
"planning_time_ms": 115,
"retrieval_time_ms": 525,
"response_generation_time_ms": 182,
"context_size": 2951,
"answer_text": "[simulated-4o-mini-profile] route=hybrid_store_plus_live; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
},
{
"question_id": "Q35",
"question_text": "Проверить документы июня.",
"question_class": "ambiguous_fuzzy",
"expected_route": "store_feature_risk",
"actual_route": "store_feature_risk",
"sources_used": [
"feature_store",
"risk_store",
"canonical_store"
],
"refresh_needed": false,
"latency_ms": 405,
"planning_time_ms": 86,
"retrieval_time_ms": 156,
"response_generation_time_ms": 163,
"context_size": 2102,
"answer_text": "[simulated-4o-mini-profile] route=store_feature_risk; answer synthesized from June-2020 slice + current stores.",
"answer_quality_assessment": "acceptable",
"route_quality_assessment": "good",
"issues_detected": [],
"recommended_fix": "No action required."
}
]
}
docs/ARCH/validation_run_2026-03-23/llm_like_simulation_profile.md
+27
View File
@@ -0,0 +1,27 @@
# LLM-like Simulation Profile
Simulation mode: `4o-mini-like` (controlled emulation)
## Constraints
- Store-first retrieval policy.
- Compact planning and bounded context.
- Limited live calls for drill-down only.
- Avoid expensive heavy live scans.
## Route timing baseline (ms)
| Route | Planning | Retrieval | Generation | Context |
| --- | --- | --- | --- | --- |
| live_mcp_drilldown | 95 | 780 | 180 | 2900 |
| store_canonical | 70 | 170 | 130 | 1700 |
| store_feature_risk | 82 | 190 | 150 | 2200 |
| hybrid_store_plus_live | 112 | 560 | 170 | 3050 |
| batch_refresh_then_store | 135 | 1240 | 210 | 3600 |
## Active run context
- Slice window: `2020-06`
- Refresh latest run: `30b2a2da4d824e0b81c2fb263cb9b64b`
- Feature latest run: `2d2dc33e509e4f5681b64217673dad09`
- Risk latest run: `d4833f6aa39f4bb5b4897c3d48caa843`
docs/ARCH/validation_run_2026-03-23/ontology_mapping_audit.md
+50
View File
@@ -0,0 +1,50 @@
# Ontology & Mapping Audit
## Core metrics
| Metric | Value |
| --- | --- |
| entity_classes_total | 42 |
| covered_entity_classes | 33 |
| uncovered_entity_classes | 9 |
| relation_types_total | 1 |
| correctly_typed_relations | 1602 |
| unknown_relations | 1016 |
| conflicting_mappings_count | 0 |
| link_coverage_pct | 100.0 |
| semantic_coverage_pct | 61.1917 |
## Top problematic source entity types
| Source entity | Unknown relations |
| --- | --- |
| AccumulationRegister_НДСПредъявленный_RecordType | 130 |
| Document_СписаниеСРасчетногоСчета | 116 |
| DocumentJournal_ЖурналОпераций | 114 |
| AccumulationRegister_НДСЗаписиКнигиПродаж_RecordType | 92 |
| DocumentJournal_БанковскиеВыписки | 90 |
| DocumentJournal_ДокументыПоставщиков | 90 |
| Document_РеализацияТоваровУслуг | 60 |
| Document_ПоступлениеТоваровУслуг | 50 |
| DocumentJournal_ДокументыПокупателей | 32 |
| AccumulationRegister_НДФЛРасчетыСБюджетом_RecordType | 28 |
| Document_СчетФактураВыданный | 25 |
| AccumulationRegister_НДСЗаписиКнигиПокупок_RecordType | 24 |
## Top problematic relation fields
| Source field | Unknown relations |
| --- | --- |
| Ответственный_Key | 187 |
| Ref | 168 |
| Recorder | 147 |
| Поставщик_Key | 78 |
| ФизЛицо_Key | 49 |
| Информация | 49 |
| Покупатель_Key | 46 |
| Валюта_Key | 34 |
| СтатьяДвиженияДенежныхСредств_Key | 34 |
| ПодразделениеДт_Key | 29 |
| ОбособленноеПодразделение_Key | 18 |
| Склад_Key | 16 |
docs/ARCH/validation_run_2026-03-23/orchestration_policy_spec.md
+37
View File
@@ -0,0 +1,37 @@
# Orchestration Policy Spec
## Decision tree
- exact object trace or posting chain -> `live_mcp_drilldown`
- simple factual in loaded slice -> `store_canonical`
- trend/anomaly/risk -> `store_feature_risk`
- heavy whole-slice with freshness gap -> `batch_refresh_then_store`
- low confidence fallback -> `hybrid_store_plus_live`
## Routing rules
- Prefer store answers when freshness allows.
- Use live bridge only for drill-down evidence.
- Do not run uncapped heavy live scans.
- Trigger refresh/features/risk for stale context.
- Apply retrieval/context budget before fallback.
## Source priorities
| Scenario | Priority order |
| --- | --- |
| simple_factual | canonical_store -> mcp_runtime_bridge |
| drilldown_explain | mcp_runtime_bridge -> canonical_store |
| period_trend | feature_store -> risk_store -> canonical_store |
| anomaly_control | risk_store -> feature_store -> canonical_store |
| heavy_analytical | batch_refresh_then_store -> feature_store -> risk_store |
| ambiguous_fuzzy | feature_store -> canonical_store -> mcp_runtime_bridge |
## Timeout budget (ms)
| Budget | Value |
| --- | --- |
| planning | 200 |
| retrieval_soft_limit | 1200 |
| retrieval_hard_limit | 2500 |
| response_generation | 600 |
docs/ARCH/validation_run_2026-03-23/slice_ingestion_report.md
+20
View File
@@ -0,0 +1,20 @@
# Slice Ingestion Report
Validation date: 2026-03-23T09:28:12.311411+00:00
Slice window: `2020-06` (`2020-06-01T00:00:00+00:00` -> `2020-07-01T00:00:00+00:00`)
- Snapshot file: `X:\1C\NDC_1C\logs\pre_report_snapshot_2020_2020-06.json`
- Profile file: `X:\1C\NDC_1C\logs\pre_report_activity_2020.json`
- Snapshot entities: `467`
- Snapshot links: `2618`
- Refresh run id: `30b2a2da4d824e0b81c2fb263cb9b64b`
- Entities written: `451`
- Links written: `2586`
- Checkpoints updated: `42`
- Canonical entities total: `453`
- Canonical links total: `2596`
- Feature run status: `success`
- Feature metrics written: `202`
- Risk run status: `success`
- Risk patterns written: `2`
- Risk global score: `0.608542`