ГЛОБАЛЬНЫЙ РЕФАКТОРИНГ АРХИТЕКТУРЫ - Stage 4.6: добавить метрику stage4_contract_compliance_rate в Stage1 eval и отчеты
This commit is contained in:
@@ -2709,7 +2709,26 @@ Implemented in current pass (Stage 4.5 contract observability in debug/log, 2026
|
||||
- focused assembler pack passed: `2 files / 5 tests`;
|
||||
- `npm --prefix llm_normalizer/backend run build` passed.
|
||||
|
||||
Status: In progress (Stage 4.1-4.5 completed; continue with focused wave/manual-comment quality backlog)
|
||||
Implemented in current pass (Stage 4.6 eval metric for answer-contract compliance, 2026-04-12):
|
||||
1. Added Stage 4 compliance metric to Stage 1 eval:
|
||||
- `stage4_contract_compliance_rate` in `AssistantEvalMetricVector` + rubric (`ACCOUNTANT_SCORING_RUBRIC_V01`).
|
||||
2. Integrated metric computation in Stage 1 eval runtime:
|
||||
- source: `debug.answer_contract_stage4_v1.is_stage4_shape`;
|
||||
- aggregate in `computeAssistantMetrics(...)`;
|
||||
- per-case metric projection in `metric_subscores/accountant_metrics`;
|
||||
- added denominator `stage4_contract_audited_cases_total`.
|
||||
3. Integrated metric into reporting/comparison and autorun score index:
|
||||
- Stage 1 comparison deltas now include `stage4_contract_compliance_rate`;
|
||||
- auto-run score index for `assistant_stage1` now includes this compliance signal.
|
||||
4. Regression updates:
|
||||
- `assistantContracts.test.ts`
|
||||
- `assistantEvalHarness.test.ts`
|
||||
5. Validation snapshot:
|
||||
- `assistantContracts.test.ts`: `2 passed`;
|
||||
- `assistantEvalHarness.test.ts`: `6 passed` (run with extended timeout budget);
|
||||
- `npm --prefix llm_normalizer/backend run build` passed.
|
||||
|
||||
Status: In progress (Stage 4.1-4.6 completed; continue with focused wave/manual-comment quality backlog)
|
||||
|
||||
## Stage 5 (P3): Quality Loop Driven By GUI Markup
|
||||
|
||||
|
||||
Reference in New Issue
Block a user