ГЛОБАЛЬНЫЙ РЕФАКТОРИНГ АРХИТЕКТУРЫ - Stage 4.6: добавить метрику stage4_contract_compliance_rate в Stage1 eval и отчеты

This commit is contained in:
2026-04-12 09:25:15 +03:00
parent 963f0aa372
commit ce1ebae8ec
9 changed files with 93 additions and 17 deletions
+20 -1
View File
@@ -2709,7 +2709,26 @@ Implemented in current pass (Stage 4.5 contract observability in debug/log, 2026
- focused assembler pack passed: `2 files / 5 tests`;
- `npm --prefix llm_normalizer/backend run build` passed.
Status: In progress (Stage 4.1-4.5 completed; continue with focused wave/manual-comment quality backlog)
Implemented in current pass (Stage 4.6 eval metric for answer-contract compliance, 2026-04-12):
1. Added Stage 4 compliance metric to Stage 1 eval:
- `stage4_contract_compliance_rate` in `AssistantEvalMetricVector` + rubric (`ACCOUNTANT_SCORING_RUBRIC_V01`).
2. Integrated metric computation in Stage 1 eval runtime:
- source: `debug.answer_contract_stage4_v1.is_stage4_shape`;
- aggregate in `computeAssistantMetrics(...)`;
- per-case metric projection in `metric_subscores/accountant_metrics`;
- added denominator `stage4_contract_audited_cases_total`.
3. Integrated metric into reporting/comparison and autorun score index:
- Stage 1 comparison deltas now include `stage4_contract_compliance_rate`;
- auto-run score index for `assistant_stage1` now includes this compliance signal.
4. Regression updates:
- `assistantContracts.test.ts`
- `assistantEvalHarness.test.ts`
5. Validation snapshot:
- `assistantContracts.test.ts`: `2 passed`;
- `assistantEvalHarness.test.ts`: `6 passed` (run with extended timeout budget);
- `npm --prefix llm_normalizer/backend run build` passed.
Status: In progress (Stage 4.1-4.6 completed; continue with focused wave/manual-comment quality backlog)
## Stage 5 (P3): Quality Loop Driven By GUI Markup