feat(perception): gate sealed truth evaluation

This commit is contained in:
DCCONSTRUCTIONS
2026-07-29 14:34:33 +03:00
parent 1e1b7fe43d
commit 9db39347bd
9 changed files with 2296 additions and 0 deletions
@@ -0,0 +1,128 @@
# E48E49 detector truth gate implementation
Date: 2026-07-29
Status: executable gate ready; no E48 or E49 result exists
## Purpose
E46 prepared a references-only detector Truth Island and E47 froze two
prediction candidates before truth reveal. E48E49 make the remaining boundary
executable before either reviewer submission exists:
1. E48 accepts exactly two completed independent blind reviews and one explicit
adjudication.
2. E48 seals only adjudicated truth and never reads E47 prediction content.
3. E49 refuses to run without an accepted E48 seal.
4. E49 then joins the exact E47 prediction generation and computes the frozen
detector metrics without selecting a winner automatically.
The implementation does not fabricate reviewer submissions, adjudication,
truth, metrics or a candidate decision.
## E48 review and seal contract
Each reviewer submission must use
`missioncore.e48-independent-detector-review/v1` and preserve all 32 E46 image
identities in their original order. Reviewer IDs must be different opaque
identifiers. Every submission explicitly attests that candidate identity,
prelabels, predictions and scores were not seen.
Every image must end in `reviewed` with:
- a required hard-negative decision;
- every task-relevant object represented by `object_id`, target class and
`xyxy` box in the original 800×600 coordinate system;
- required `occluded` and `truncated` flags;
- no confidence score, prediction reference or unknown field.
The adjudication document uses
`missioncore.e48-detector-review-adjudication/v1`, binds the canonical SHA-256
of both reviewer submissions, covers every image, and explicitly accepts that
all disagreements are resolved. Its sealed time may not predate the immutable
E47 prediction freeze.
The E48 sealer rejects:
- duplicate reviewer identities;
- missing, reordered or changed source-image identities;
- incomplete frame coverage;
- hidden model/prediction fields;
- unknown classes, duplicate object IDs or invalid boxes;
- a hard-negative flag that conflicts with object presence;
- incomplete review/adjudication acceptance;
- review hashes that do not match the adjudication;
- an E47 manifest bound to another Truth Island.
E48 reads the E47 manifest identity only. It does not open
`candidate-predictions.jsonl`. A successful generation records this boundary
as `prediction_content_read_by_sealer=false`.
Human independence and blindness remain explicit signed-process attestations
bound to distinct opaque identities; the code cannot prove a person's identity
or what they saw outside the controlled package.
## E49 frozen evaluation
E49 accepts only:
- the exact E46 Truth Island;
- an accepted `sealed-adjudicated-independent-truth` E48 generation;
- the exact E47 prediction freeze referenced by E48;
- a content-addressed valid-FOV mask with the same calibration SHA, camera slot
and 800×600 resolution.
It computes:
- COCO-style 101-point interpolated AP averaged over IoU 0.50:0.05:0.95;
- AP50 and AP75;
- AR100;
- per-class AP and recall at IoU 0.50;
- combined person/vehicle miss rate at IoU 0.50;
- false-large-box rate for boxes occupying at least 25% of the image;
- valid-FOV leakage as prediction-box centres outside the calibrated mask;
- temporal class-count flicker over adjacent frames in each frozen E46 clip.
The valid-FOV metric is deliberately named and bounded: it is box-centre
admission, not full box-area or mask leakage. Temporal flicker is class-count
stability because the frozen candidates do not publish track identities.
E49 always leaves `candidate_winner_selected=false` and
`model_retraining_authorized=false`. Candidate acceptance remains a separate
explicit product decision after reviewing all metrics and limitations.
## Execution
After two real reviewer files and an adjudication exist:
```text
python experiments/perception/run_e48_detector_truth_seal.py \
--truth-island-root <E46> \
--prediction-freeze-root <E47> \
--reviewer-a <review-a.json> \
--reviewer-b <review-b.json> \
--adjudication <adjudication.json> \
--output-root .runtime/compute-experiments/e48/results
```
Only after E48 succeeds:
```text
python experiments/perception/run_e49_detector_truth_evaluation.py \
--truth-island-root <E46> \
--truth-seal-root <E48> \
--prediction-freeze-root <E47> \
--valid-fov-root <valid-fov-generation> \
--output-root .runtime/compute-experiments/e49/results
```
No real command is run now because the two reviewer submissions and
adjudication do not exist.
## Resource and authority boundary
The tooling is pure local validation and metric code. It starts no Docker
container, model, Worker 006 job, video decoder, browser session, network
mutation or second Mission Core service.
Navigation, safety and command authority remain false.
@@ -0,0 +1,49 @@
#!/usr/bin/env python3
"""Seal completed E46 independent reviews into immutable E48 truth."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
from k1link.compute.e48_detector_truth_seal import (
build_e48_detector_truth_seal,
)
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--truth-island-root", type=Path, required=True)
parser.add_argument("--prediction-freeze-root", type=Path, required=True)
parser.add_argument("--reviewer-a", type=Path, required=True)
parser.add_argument("--reviewer-b", type=Path, required=True)
parser.add_argument("--adjudication", type=Path, required=True)
parser.add_argument("--output-root", type=Path, required=True)
args = parser.parse_args()
result = build_e48_detector_truth_seal(
truth_island_root=args.truth_island_root,
prediction_freeze_root=args.prediction_freeze_root,
reviewer_a_path=args.reviewer_a,
reviewer_b_path=args.reviewer_b,
adjudication_path=args.adjudication,
output_root=args.output_root,
)
print(
json.dumps(
{
"result_id": result["result_id"],
"result_root": str(result["result_root"]),
"status": result["report"]["status"],
"frame_count": result["report"]["frame_count"],
"object_count": result["report"]["object_count"],
},
ensure_ascii=False,
sort_keys=True,
)
)
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""Evaluate exact E47 predictions after the E48 truth seal."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
from k1link.compute.e49_detector_truth_evaluation import (
build_e49_detector_truth_evaluation,
)
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--truth-island-root", type=Path, required=True)
parser.add_argument("--truth-seal-root", type=Path, required=True)
parser.add_argument("--prediction-freeze-root", type=Path, required=True)
parser.add_argument("--valid-fov-root", type=Path, required=True)
parser.add_argument("--output-root", type=Path, required=True)
args = parser.parse_args()
result = build_e49_detector_truth_evaluation(
truth_island_root=args.truth_island_root,
truth_seal_root=args.truth_seal_root,
prediction_freeze_root=args.prediction_freeze_root,
valid_fov_root=args.valid_fov_root,
output_root=args.output_root,
)
print(
json.dumps(
{
"result_id": result["result_id"],
"result_root": str(result["result_root"]),
"status": result["report"]["status"],
"candidates": result["report"]["candidates"],
},
ensure_ascii=False,
sort_keys=True,
)
)
return 0
if __name__ == "__main__":
raise SystemExit(main())