# v2.0.1 Clarification Eval Plan

## Objective

Measure clarification-threshold quality for `normalizer_v2_0_1` without degrading schema/scope stability.

## Dataset

Target: 15-20 cases.

Recommended mix:
- 10 in-scope one-step operational questions that should not require clarification.
- 5 truly ambiguous questions that should require clarification.
- 3-5 mixed/out-of-scope questions.

## Labels

Use per-case expected field:
- `expected.clarification_required: boolean`

Optional:
- `expected.expected_execution_readiness`

## Metrics

Primary:
- `clarification_precision`
- `clarification_recall`
- `false_clarification_rate`
- `executable_with_soft_assumptions_rate`

Guardrails:
- `schema_validation_pass_rate`
- `scope_in_scope_rate`
- `out_of_scope_fragment_rate`
- `routed_fragment_rate`

## Acceptance Targets

Minimum:
- `clarification_required_rate <= 50%` on one-step in-scope set
- lower `false_clarification_rate` vs current baseline
- no regression in schema/scope guardrails

Good:
- `clarification_required_rate <= 35%`

Very good:
- `clarification_required_rate <= 25%`

