ДОМЕНЫ - ВОПРОСЫ - ОРРКЕСТРАЦИЯ - БАЗА -Зафиксировать в orchestration loop правило domain enablement для новых неразмеченных доменов

This commit is contained in:
dctouch 2026-04-13 20:04:43 +03:00
parent 8e16bc1f01
commit c2ac0c610b
8 changed files with 112 additions and 0 deletions

View File

@ -29,6 +29,8 @@ Read `references/repo_runtime_map.md` before the first real cycle.
Use these repo-native capture paths:
- automated capture: `python scripts/domain_case_loop.py run-case ...`
- import existing technical export: `python scripts/domain_case_loop.py import-export ...`
- `run-case` defaults to the repo's live local profile: `local / qwen2.5-14b-instruct-1m / http://127.0.0.1:1234/v1`
- override with `--llm-provider`, `--llm-model`, `--llm-base-url`, `--llm-api-key` when needed
## Workflow
@ -69,6 +71,11 @@ Spawn `domain_analyst` and provide:
Require a full verdict using `references/verdict_template.md`.
The verdict must explicitly say whether the case is:
- an existing in-contour regression;
- a missing route/intent/capability inside project scope;
- a true out-of-scope request.
### Step 4 - Domain patch
Spawn `domain_coder` with:
@ -80,6 +87,7 @@ Require:
- a minimal patch
- zero architecture drift
- rerun after changes
- if the domain is in project scope but outside the current contour, convert the verdict into capability enablement work instead of closing the case as unsupported
### Step 5 - Rerun
@ -104,6 +112,8 @@ Write `final_status.md` with one of:
- blocked
- needs_exact_capability
`needs_exact_capability` is the default status when the business/domain request is valid for the project, but the current contour is missing the route, intent, capability, or domain bootstrap needed to answer it.
Accepted requires:
- quality score >= 80
- no unresolved P0 defects
@ -114,6 +124,7 @@ Accepted requires:
- Do not count heuristic candidates as confirmed business answers.
- If exact data should exist in 1C/MCP, prefer exact route work over prompt cosmetics.
- If exact data does not exist yet in the reachable contour, return a technical insufficiency with a crisp blocker.
- If the user case belongs to a project-relevant domain but is outside the current contour, do not treat that as a terminal rejection. Treat it as domain enablement work and record the missing route/intent/capability explicitly.
- Never fabricate 1C data.
- Keep domain fixes minimal and localized.
- Preserve successful baseline scenarios.
@ -127,6 +138,7 @@ For this repository:
- analyst output must be detailed and business-readable;
- answers should be suitable for product hardening, not just debugging notes;
- machine-readable turn artifacts are first-class inputs for analysis.
- New user domains may be unmarked in the current repo. Missing markup is expected and should be handled as enablement, not as a reason to stop the loop.
## Recommended artifact set

View File

@ -15,6 +15,12 @@
## Expected result mode
- confirmed_balance / confirmed_tax_liability / partial / technical_insufficiency / other
## Contour status
- in_contour / outside_current_contour / unknown
## Enablement hypothesis
- missing_route / missing_intent / missing_capability / missing_data_access / unknown
## Constraints
- no architecture changes
- 1C/MCP first

View File

@ -6,4 +6,6 @@
- Не считать heuristic ответ продуктовым успехом.
- Математика вне 1С допустима только как детерминированный постпроцесс над уже подтвержденными фактами.
- Analyst read-only, Coder implementation-focused.
- Если домен или маршрут еще не прокинут, но кейс лежит внутри целевого project scope, это считается работой на enablement, а не поводом закрыть кейс как unsupported.
- Для новых неразмеченных доменов outer loop должен явно фиксировать, чего не хватает: intent, route, capability, data access или bootstrap слоя.
- Accepted требует score >= 80, zero unresolved P0 и отсутствия silent fallback masking.

View File

@ -18,4 +18,6 @@ Rules:
- Use `.codex/skills/domain-case-loop` for repeatable domain hardening loops on one concrete case.
- Preserve current architecture: domain loop may automate capture, review, rerun, and artifact storage, but must not rewrite runtime foundations.
- Prefer machine-readable case artifacts in `artifacts/domain_runs/<case_id>/`, especially `baseline_turn.json` / `rerun_turn.json`, over ad hoc prose-only summaries.
- If a case falls outside the current routed contour because the route/intent/capability is not wired yet, treat it as domain enablement work for this project, not as automatic out-of-scope rejection.
- For new unmarked domains, `needs_exact_capability` means "bootstrap or extend the contour" rather than "close the case as unsupported".
- A case can be marked `accepted` only when analyst verdict is at least `80/100`, no unresolved `P0` remains, and the rerun does not mask heuristic output as confirmed.

View File

@ -46,6 +46,9 @@
Использует живой backend:
- по умолчанию helper запускает кейс через `local / qwen2.5-14b-instruct-1m / http://127.0.0.1:1234/v1`;
- для переопределения доступны `--llm-provider`, `--llm-model`, `--llm-base-url`, `--llm-api-key`.
```powershell
python scripts/domain_case_loop.py run-case `
--domain open_contracts `
@ -103,3 +106,23 @@ python scripts/domain_case_loop.py import-export `
- `partial`
- `blocked`
- `needs_exact_capability`
`needs_exact_capability` здесь означает не "закрыть кейс как чужой", а "доменный запрос валиден для проекта, но текущий контур еще не умеет его отрабатывать точно".
## Политика для новых и неразмеченных доменов
Outer loop должен считать нормальным, что пользователь будет приносить кейсы из доменов, которые еще не размечены в текущем runtime.
Если baseline показывает, что:
1. вопрос лежит внутри целевого project scope;
2. 1С/MCP/данные проекта концептуально относятся к этому домену;
3. ответ не получен из-за отсутствующего intent/route/capability/bootstrap;
то кейс нельзя автоматически считать `out_of_scope`.
Такой кейс должен переходить в режим domain enablement:
- явно зафиксировать, чего не хватает: `intent`, `route`, `capability`, `data access`, `contour bootstrap`;
- сформировать минимальную задачу на прокидывание контура;
- после этого повторно прогонять baseline/rerun уже как продуктовый кейс.

View File

@ -0,0 +1,22 @@
{
"suite_id": "assistant_autogen_runtime_job-Bv_o7Vircc",
"suite_version": "0.1.0",
"schema_version": "assistant_autogen_runtime_v0_1",
"scenario_count": 1,
"case_ids": [
"AUTO-001"
],
"cases": [
{
"case_id": "AUTO-001",
"scenario_tag": "autogen_runtime",
"question_type": "direct",
"broadness_level": "medium",
"turns": [
{
"user_message": "Какие товары сейчас лежат на складе"
}
]
}
]
}

View File

@ -0,0 +1,22 @@
{
"suite_id": "assistant_autogen_runtime_job-vHloW0L_3W",
"suite_version": "0.1.0",
"schema_version": "assistant_autogen_runtime_v0_1",
"scenario_count": 1,
"case_ids": [
"AUTO-001"
],
"cases": [
{
"case_id": "AUTO-001",
"scenario_tag": "autogen_runtime",
"question_type": "direct",
"broadness_level": "medium",
"turns": [
{
"user_message": "Какие товары лежат на складе сейчас или на любую выбранную дату"
}
]
}
]
}

View File

@ -18,6 +18,13 @@ DEFAULT_ARTIFACTS_ROOT = REPO_ROOT / "artifacts" / "domain_runs"
DEFAULT_SESSIONS_DIR = REPO_ROOT / "llm_normalizer" / "data" / "assistant_sessions"
DEFAULT_REPORTS_DIR = REPO_ROOT / "llm_normalizer" / "reports"
DEFAULT_BACKEND_URL = "http://127.0.0.1:8787"
DEFAULT_PROMPT_VERSION = "address_query_runtime_v1"
DEFAULT_LLM_PROVIDER = "local"
DEFAULT_LLM_MODEL = "qwen2.5-14b-instruct-1m"
DEFAULT_LLM_BASE_URL = "http://127.0.0.1:1234/v1"
DEFAULT_LLM_API_KEY = ""
DEFAULT_TEMPERATURE = 0.0
DEFAULT_MAX_OUTPUT_TOKENS = 900
TECH_SECTION_HEADER = "### technical_debug_payload_json"
@ -444,6 +451,15 @@ def handle_run_case(args: argparse.Namespace) -> int:
)
payload: dict[str, Any] = {
"normalizeConfig": {
"llmProvider": args.llm_provider,
"apiKey": args.llm_api_key,
"model": args.llm_model,
"baseUrl": args.llm_base_url,
"temperature": args.temperature,
"maxOutputTokens": args.max_output_tokens,
"promptVersion": args.prompt_version,
},
"eval_target": "assistant_stage1",
"questions": [args.question],
"useMock": bool(args.use_mock),
@ -564,6 +580,13 @@ def build_parser() -> argparse.ArgumentParser:
run_case.add_argument("--output-root", default=str(DEFAULT_ARTIFACTS_ROOT))
run_case.add_argument("--sessions-dir", default=str(DEFAULT_SESSIONS_DIR))
run_case.add_argument("--reports-dir", default=str(DEFAULT_REPORTS_DIR))
run_case.add_argument("--prompt-version", default=DEFAULT_PROMPT_VERSION)
run_case.add_argument("--llm-provider", default=DEFAULT_LLM_PROVIDER, choices=["openai", "local"])
run_case.add_argument("--llm-model", default=DEFAULT_LLM_MODEL)
run_case.add_argument("--llm-base-url", default=DEFAULT_LLM_BASE_URL)
run_case.add_argument("--llm-api-key", default=DEFAULT_LLM_API_KEY)
run_case.add_argument("--temperature", type=float, default=DEFAULT_TEMPERATURE)
run_case.add_argument("--max-output-tokens", type=int, default=DEFAULT_MAX_OUTPUT_TOKENS)
run_case.add_argument("--timeout-seconds", type=int, default=300)
run_case.add_argument("--poll-interval-seconds", type=float, default=1.5)
run_case.add_argument("--expected-capability")