# LLM Normalizer v1.1.2.1 Eval Run

- run_id: eval-h3k8TyTFuu
- timestamp: 2026-03-23T16:30:36.413Z
- mode: single-pass-strict
- use_mock: false
- cases_total: 30
- prompt_version: normalizer_v1_1_2_1

## Metrics vs Baseline

| Metric | Current | Baseline | Delta |
|---|---:|---:|---:|
| schema_validation_pass_rate | 100.00% | 100.00% | 0.00 |
| intent_class_accuracy | 70.00% | 72.73% | -2.73 |
| route_hint_accuracy | 80.00% | 90.91% | -10.91 |
| causal_flag_accuracy | 60.00% | 81.82% | -21.82 |
| high_confidence_error_rate | 3.33% | 9.09% | -5.76 |

## Class Accuracy

| Intent class | Passed/Total | Accuracy |
|---|---:|---:|
| cross_entity | 14/14 | 100.00% |
| anomaly_probe | 3/6 | 50.00% |
| heavy_analytical | 1/5 | 20.00% |
| rule_based_account_control | 3/5 | 60.00% |

## Budget

- requests_total: 30
- retries_used: 0

## Mismatches

- V1121-B1-06: expected(anomaly_probe / store_feature_risk) -> actual(cross_entity / store_feature_risk). Route chosen correctly, but intent_class drifted into a neighboring taxonomy bucket.
- V1121-B2-01: expected(heavy_analytical / batch_refresh_then_store) -> actual(cross_entity / hybrid_store_plus_live). Both intent and route misclassified; likely lexical ambiguity in causal vs risk wording.
- V1121-B2-03: expected(heavy_analytical / batch_refresh_then_store) -> actual(cross_entity / store_feature_risk). Both intent and route misclassified; likely lexical ambiguity in causal vs risk wording.
- V1121-B2-06: expected(heavy_analytical / batch_refresh_then_store) -> actual(cross_entity / hybrid_store_plus_live). Both intent and route misclassified; likely lexical ambiguity in causal vs risk wording.
- V1121-B3-01: expected(cross_entity / hybrid_store_plus_live) -> actual(cross_entity / store_feature_risk). Intent understood, but route_hint selected a weaker execution route.
- V1121-B3-02: expected(cross_entity / hybrid_store_plus_live) -> actual(cross_entity / store_feature_risk). Intent understood, but route_hint selected a weaker execution route.
- V1121-B3-03: expected(anomaly_probe / store_feature_risk) -> actual(anomaly_probe / store_feature_risk). Causal flags are inconsistent with expected relationship depth.
- V1121-B3-05: expected(heavy_analytical / batch_refresh_then_store) -> actual(cross_entity / hybrid_store_plus_live). Both intent and route misclassified; likely lexical ambiguity in causal vs risk wording.
- V1121-B4-02: expected(anomaly_probe / store_feature_risk) -> actual(anomaly_probe / store_feature_risk). Causal flags are inconsistent with expected relationship depth.
- V1121-B5-01: expected(rule_based_account_control / store_feature_risk) -> actual(cross_entity / store_feature_risk). Route chosen correctly, but intent_class drifted into a neighboring taxonomy bucket.
- V1121-B5-02: expected(anomaly_probe / store_feature_risk) -> actual(anomaly_probe / store_feature_risk). Causal flags are inconsistent with expected relationship depth.
- V1121-B5-03: expected(anomaly_probe / store_feature_risk) -> actual(cross_entity / store_feature_risk). Route chosen correctly, but intent_class drifted into a neighboring taxonomy bucket.
- V1121-B6-02: expected(anomaly_probe / store_feature_risk) -> actual(cross_entity / store_feature_risk). Route chosen correctly, but intent_class drifted into a neighboring taxonomy bucket.
- V1121-B7-02: expected(rule_based_account_control / store_feature_risk) -> actual(anomaly_probe / store_feature_risk). Route chosen correctly, but intent_class drifted into a neighboring taxonomy bucket.

## Bad Confidence Cases

- V1121-B2-06: confidence=high, intent_match=false, route_match=false
