diff --git a/docs/13_LIDAR_WORKER_PRODUCT_AND_ROADMAP.md b/docs/13_LIDAR_WORKER_PRODUCT_AND_ROADMAP.md index b804623..a9834ee 100644 --- a/docs/13_LIDAR_WORKER_PRODUCT_AND_ROADMAP.md +++ b/docs/13_LIDAR_WORKER_PRODUCT_AND_ROADMAP.md @@ -718,6 +718,18 @@ Execution was strictly sequential through E35: E30 determined what E31 was allowed to change; E31 determined the E32 profile; E32 determined the E33 runtime input; E32/E33 then bound the E34 layer; E32–E34 then bound E35. Exact nominal and degradation accounting are closed, so A8 is complete. + +R1 perception quality is now measured by two immutable Worker 006 results. +E38 established the baseline at 82.2% presence, 81.5% geometry association, +95.2% freshness and 14 high-severity failures. E39 result +`e39-perception-refinement-2fd253940c9d9a2fd3b3237f3f0932f81a9f69741935d793771243d5779af464` +used a fixed package-bound camera + LiDAR feature projection and improved +presence and geometry association to 84.9% while reducing high-severity +failures to 8. Freshness remained above target at 93.2%. Its development +cross-validation exceeded 90% but did not predict the sealed result, so R1 +remains open. The next iteration is development-only grouped time/scene +qualification with a source-coordinate-free representation; validation is not +opened for item-level tuning. E36 is the first generalization gate. A separate product decision follows: either keep the result as operator/shadow evidence, or start L5 occupied-space integration. No LAB in this cycle can enable navigation, commands or safety diff --git a/docs/16_ARCHITECTURE_AUDIT_EXECUTION_ROADMAP.md b/docs/16_ARCHITECTURE_AUDIT_EXECUTION_ROADMAP.md index 3a64d78..21cbaf0 100644 --- a/docs/16_ARCHITECTURE_AUDIT_EXECUTION_ROADMAP.md +++ b/docs/16_ARCHITECTURE_AUDIT_EXECUTION_ROADMAP.md @@ -246,9 +246,18 @@ The first R1 baseline is E38 immutable Worker 006 result Its sealed 146-item validation measures presence at 82.2%, geometry association at 81.5% and freshness at 95.2%, with 100% accounting, zero false-free claims and 14 high-severity failures. R1 therefore remains open; -the next bounded iteration targets detector/background presence and +the next bounded iteration targeted detector/background presence and geometry-only association without changing the E37 validation set. +E39 completed that refinement in immutable Worker 006 result +`e39-perception-refinement-2fd253940c9d9a2fd3b3237f3f0932f81a9f69741935d793771243d5779af464`. +Five-fold development CV crossed 90% for all three dimensions without +validation-label access. Sealed validation improved presence to 84.9%, +geometry association to 84.9% and reduced high-severity failures from 14 to +8; freshness remained above target at 93.2%. The aggregate gate remains +failed. R1 now advances to a harder grouped time/scene development protocol +and a source-coordinate-free representation before any new sealed evaluation. + - [x] Reproduce all 4,489 immutable E29 frames with the exact frozen profile before applying E31/E30 changes. - [x] Apply only the admitted semantic self-mask, two exact geometry diff --git a/docs/20_RAVNOVES00_REFERENCE_SOURCE_PRODUCT_PLAN.md b/docs/20_RAVNOVES00_REFERENCE_SOURCE_PRODUCT_PLAN.md index b79d4a1..a17a4cb 100644 --- a/docs/20_RAVNOVES00_REFERENCE_SOURCE_PRODUCT_PLAN.md +++ b/docs/20_RAVNOVES00_REFERENCE_SOURCE_PRODUCT_PLAN.md @@ -78,6 +78,22 @@ high-severity validation cases fail, so R1 is not accepted. The next R1 iteration is bounded to detector/background presence, geometry-only association and the camera-only freshness tail without changing validation. +The second R1 measurement is immutable result +`e39-perception-refinement-2fd253940c9d9a2fd3b3237f3f0932f81a9f69741935d793771243d5779af464`. +Its fixed robust three-neighbour refinement used exact camera-crop, LiDAR +shape and projection features and was selected only through five-fold +development cross-validation. Development reached 90.3% presence, 90.6% +geometry association and 96.5% freshness without validation-label access. +Sealed validation reached 84.9%, 84.9% and 93.2% respectively. Accounting +remains 100%, false-free claims remain zero and high-severity failures fall +from 14 to 8, but R1 is still not accepted. + +The next R1 iteration must address the development-to-validation gap before +adding model complexity: grouped time/scene development folds replace +item-hash folds, and a source-coordinate-free representation is compared under +that harder protocol. No individual E39 validation label may be used for +diagnosis, fitting or selection. + ## Experiment rules 1. RAVNOVES00 remains immutable and is never relabelled as a LAB result. diff --git a/experiments/perception/LAB_E39_REPORT_2026-07-28.md b/experiments/perception/LAB_E39_REPORT_2026-07-28.md new file mode 100644 index 0000000..1014b39 --- /dev/null +++ b/experiments/perception/LAB_E39_REPORT_2026-07-28.md @@ -0,0 +1,167 @@ +# LAB E39 — RAVNOVES00 R1 perception refinement + +Date: 2026-07-28 + +Status: completed refinement; R1 quality gate not passed + +## Why this LAB exists + +E38 established the first immutable R1 baseline on the frozen E37 contract: +82.2% presence, 81.5% geometry association, 95.2% freshness and 14 +high-severity failures. E39 tests whether a richer exact camera + LiDAR +representation can close that gap without fitting or selecting against the +sealed validation labels. + +The experiment remains source-scoped to RAVNOVES00. It does not evaluate +navigation, planning, commands, safety or transfer to another route or rig. + +## Immutable inputs + +| Input | Identity | +| --- | --- | +| Physical source | `20260720T065719Z_viewer_live` (`RAVNOVES00`) | +| Frozen E37 contract | `e37-ravnoves-acceptance-01b1efd586f747341c712d82f0907b39436a6f91ae92b1dfae987eca05fd8344` | +| E37 item digest | `c6e4f474f59867ce0cc86825193043245c227106400cc216a2dac0645dca54ab` | +| E30 materialization | `e30-materialization-841af926d8d28ab93538c46d8f31278a2234c4d1c12c7dc4dc296b249d59735a` | +| E30 item digest | `827b498c7b7ab520cc3998e288134ebb5a2e8ebba0532d5cc3905e1a4460b0a7` | +| E39 profile | `e39-ravnoves00-r1-development-knn/v1` | +| Profile digest | `805828c1eee7b680325c0e72acca0b6a700ead1d267b2745ad7e5227ad6dce34` | + +The denominator is unchanged: 486 reviewed cases split into 340 development +and 146 sealed validation cases. + +## Method + +E39 derives 262 deterministic features for each immutable E37 case: + +- source frame and case ordinal; +- source stratum, range and geometry categories; +- LiDAR point counts, axis quantiles, covariance ratios and class fractions; +- projected pixel, depth and height distributions; +- a fixed 6×6 RGB camera crop plus luminance quantiles. + +Features are robustly scaled from development statistics. Presence is +predicted by a fixed three-neighbour classifier. Geometry association and +freshness are projected from predicted presence plus the immutable source +stratum; they are not independent learned models in E39. + +The fixed profile is admitted only if deterministic five-fold development +cross-validation reaches all three 90% targets. The split seed is +`e39-dev-cv`. Validation labels are neither read by feature construction nor +used for fitting, threshold selection or model selection. After development +qualification, the frozen model is evaluated once on all 146 sealed cases. + +The package includes a digest-bound feature cache so the pinned Worker image +requires only NumPy at execution time. Exact camera images and point arrays +remain packaged for audit. Cache manifest, feature names, arrays, package +contents and result artifacts are independently hashed. + +## Development qualification + +| Dimension | Correct | Accuracy | Target | Development gate | +| --- | ---: | ---: | ---: | --- | +| Presence | 307 / 340 | 90.3% | >= 90% | pass | +| Geometry association | 308 / 340 | 90.6% | >= 90% | pass | +| Freshness | 328 / 340 | 96.5% | >= 90% | pass | + +This qualification authorized one sealed evaluation. It did not authorize a +release decision by itself. + +## Worker execution + +| Property | Value | +| --- | --- | +| Worker | `DESKTOP-OPJ8J04` (`Worker 006`) | +| Worker package | `e39-worker-package-4902eafcc3fff1750d807775214d4262d777ad628a3532128b2cb55625f46b25` | +| Package materialization | 967 files, 65,139,625 bytes | +| Container | `ndc-mission-core-e39-refinement` | +| Pinned image | `nvcr.io/nvidia/tritonserver:26.06-py3@sha256:58df7489c3f2276f9591d500a012dee03e23d35543ce3c390b4c001e6bf90794` | +| Network | disabled | +| Limits | 2 CPU, 1 GiB memory and swap, 128 PIDs | +| Filesystem | read-only runtime, explicit output mount, 64 MiB no-exec tmpfs | +| Authority | commands false; navigation/safety acceptance false | + +The run was executed sequentially to preserve the 18 GiB operator machine and +the shared Worker. The immutable result was copied back from Worker 006 and +validated against its manifest and artifact digests. + +## Official result + +Result identity: + +`e39-perception-refinement-2fd253940c9d9a2fd3b3237f3f0932f81a9f69741935d793771243d5779af464` + +| Dimension | Correct | Accuracy | Target | Gate | +| --- | ---: | ---: | ---: | --- | +| Presence | 124 / 146 | 84.9% | >= 90% | fail | +| Geometry association | 124 / 146 | 84.9% | >= 90% | fail | +| Freshness | 136 / 146 | 93.2% | >= 90% | pass | +| Evidence accounting | 146 / 146 | 100% | 100% | pass | +| False-free claims | 0 | — | 0 | pass | +| High-severity failures | 8 | — | 0 | fail | + +The R1 quality gate is not passed. Blocking checks are presence target, +geometry-association target and zero high-severity failures. + +### Change from E38 + +| Measure | E38 | E39 | Change | +| --- | ---: | ---: | ---: | +| Presence | 82.2% | 84.9% | +2.7 percentage points | +| Geometry association | 81.5% | 84.9% | +3.4 percentage points | +| Freshness | 95.2% | 93.2% | -2.0 percentage points | +| High-severity failures | 14 | 8 | -6 | + +E39 is a real improvement in presence, geometry and severe-error count, but it +does not satisfy the acceptance contract. The development-to-validation gap +is approximately 5.4 percentage points for presence and 5.7 points for +geometry association. + +### Remaining weak strata + +- Presence and geometry association are both 79.5% on `camera-only`. +- Presence and geometry association are both 78.9% on `geometry-only`. +- Freshness is 79.5% on `camera-only`. +- `conflict` is 100%, but contains only 11 validation cases and must not mask + the larger weak strata. + +The observed CV-to-validation gap is consistent with insufficiently +independent development folds and an overly source-local representation, +especially because source-frame coordinates are explicit features. This is an +engineering interpretation, not a proven causal diagnosis. + +## What E39 proves + +- Richer exact camera + LiDAR features improve E38 on the same frozen source. +- The complete Worker packaging and package-bound cache are reproducible + without modifying the pinned runtime image. +- All 146 validation cases receive terminal outcomes. +- Accounting remains 100% and no missing evidence becomes asserted free + space. +- High-severity failures fall from 14 to 8. + +## What E39 does not prove + +- Presence and geometry association do not reach 90%. +- The development CV estimate is not sufficiently predictive of sealed + validation. +- The source-coordinate-heavy representation is not route-independent. +- E39 does not prove another camera, mount, K1 unit, vegetation environment + or rover installation. +- E39 grants no navigation, command, planning or safety authority. + +## Decision and next iteration + +Keep E39 immutable as the second measured R1 result. Do not retune the +three-neighbour model against its sealed predictions. + +The next R1 iteration must be developed only on the 340 development cases and +must first replace optimistic item-hash folds with deterministic grouped +time/scene folds. It should then compare a source-coordinate-free +representation against E39 under that harder development protocol. A new +sealed evaluation is allowed only if the grouped development gate reaches all +three targets and the method is frozen under a new LAB identity. + +Individual validation labels remain unavailable for error mining or parameter +selection. Aggregate E39 metrics may be reported as historical evidence but +must not become training features. diff --git a/experiments/perception/e39_ravnoves00_r1_refinement_profile.json b/experiments/perception/e39_ravnoves00_r1_refinement_profile.json new file mode 100644 index 0000000..945f651 --- /dev/null +++ b/experiments/perception/e39_ravnoves00_r1_refinement_profile.json @@ -0,0 +1,29 @@ +{ + "authority": { + "commands_enabled": false, + "navigation_or_safety_accepted": false + }, + "model": { + "cross_validation_folds": 5, + "cross_validation_seed": "e39-dev-cv", + "feature_set": "source-scoped-camera-lidar-shape-v1", + "neighbors": 3, + "robust_clip": 20.0, + "type": "deterministic-robust-knn" + }, + "profile_id": "e39-ravnoves00-r1-development-knn/v1", + "schema_version": "missioncore.e39-perception-refinement-profile/v1", + "source": { + "acceptance_result_id": "e37-ravnoves-acceptance-01b1efd586f747341c712d82f0907b39436a6f91ae92b1dfae987eca05fd8344", + "display_name": "RAVNOVES00", + "materialization_id": "e30-materialization-841af926d8d28ab93538c46d8f31278a2234c4d1c12c7dc4dc296b249d59735a", + "session_id": "20260720T065719Z_viewer_live" + }, + "targets": { + "accounting_target": 1.0, + "freshness_target": 0.9, + "geometry_association_target": 0.9, + "maximum_false_free_claims": 0, + "presence_target": 0.9 + } +} diff --git a/experiments/perception/prepare_e39_worker_package.py b/experiments/perception/prepare_e39_worker_package.py new file mode 100644 index 0000000..6e621cf --- /dev/null +++ b/experiments/perception/prepare_e39_worker_package.py @@ -0,0 +1,440 @@ +#!/usr/bin/env python3 +"""Build an immutable camera/LiDAR E39 package for Worker 006.""" + +from __future__ import annotations + +import argparse +import hashlib +import io +import json +import os +import shutil +import uuid +from datetime import UTC, datetime +from pathlib import Path +from typing import Any + +import numpy as np + +from k1link.compute.e39_perception_refinement import ( + _FEATURE_CACHE_ARRAYS, + _FEATURE_CACHE_MANIFEST, + _FEATURE_CACHE_SCHEMA, + E39_PACKAGE_SCHEMA, + E39_PROFILE_SCHEMA, + _feature_names, + _feature_vector, +) + +_RUNTIME_FILES = { + "runtime/k1link/__init__.py": "src/k1link/__init__.py", + "runtime/k1link/compute/__init__.py": None, + "runtime/k1link/compute/e37_acceptance_contract.py": ( + "src/k1link/compute/e37_acceptance_contract.py" + ), + "runtime/k1link/compute/e39_perception_refinement.py": ( + "src/k1link/compute/e39_perception_refinement.py" + ), + "runtime/run_e39_perception_refinement.py": ( + "experiments/perception/worker/run_e39_perception_refinement.py" + ), + "runtime/Invoke-E39PerceptionRefinement.ps1": ( + "experiments/perception/worker/Invoke-E39PerceptionRefinement.ps1" + ), +} +_GENERATED_COMPUTE_INIT = ( + '"""Minimal E39 worker projection; import contract modules explicitly."""\n' +) +_ACCEPTANCE_FILES = ( + "manifest.json", + "acceptance-items.jsonl", + "acceptance-contract.json", + "run-report.json", +) +_MATERIALIZATION_FILES = ("manifest.json", "materialized-items.jsonl") + + +class E39WorkerPackageError(RuntimeError): + """The E39 package source or immutable package is invalid.""" + + +def build_e39_worker_package( + *, + repository_root: Path, + acceptance_root: Path, + materialization_root: Path, + profile_path: Path, + output_root: Path, +) -> Path: + """Build or verify one content-addressed E39 worker package.""" + + repository = repository_root.resolve(strict=True) + profile_source = profile_path.resolve(strict=True) + profile = _read_json(profile_source) + if profile.get("schema_version") != E39_PROFILE_SCHEMA: + raise E39WorkerPackageError("E39 package profile is incompatible") + acceptance = acceptance_root.resolve(strict=True) + materialization = materialization_root.resolve(strict=True) + expected_ids = { + "acceptance": profile["source"]["acceptance_result_id"], + "materialization": profile["source"]["materialization_id"], + } + if ( + acceptance.name != expected_ids["acceptance"] + or materialization.name != expected_ids["materialization"] + ): + raise E39WorkerPackageError("E39 source identity changed") + + sources: dict[str, Path | bytes | None] = {} + for target, relative in _RUNTIME_FILES.items(): + source = None if relative is None else repository / relative + if source is not None and (not source.is_file() or source.is_symlink()): + raise E39WorkerPackageError(f"E39 runtime source is invalid: {relative}") + sources[target] = source + sources["profile.json"] = profile_source + for filename in _ACCEPTANCE_FILES: + source = acceptance / filename + if not source.is_file() or source.is_symlink(): + raise E39WorkerPackageError("E39 acceptance artifact is invalid") + sources[ + f"input/acceptance/{acceptance.name}/{filename}" + ] = source + for filename in _MATERIALIZATION_FILES: + source = materialization / filename + if not source.is_file() or source.is_symlink(): + raise E39WorkerPackageError("E39 materialization artifact is invalid") + sources[ + f"input/materialization/{materialization.name}/{filename}" + ] = source + _add_materialized_evidence( + sources, + materialization=materialization, + ) + _add_feature_cache( + sources, + acceptance=acceptance, + materialization=materialization, + ) + + descriptors = [] + for relative, source in sorted(sources.items()): + payload = ( + _GENERATED_COMPUTE_INIT.encode() + if source is None + else source + if isinstance(source, bytes) + else source.read_bytes() + ) + descriptors.append( + { + "path": relative, + "byte_length": len(payload), + "sha256": hashlib.sha256(payload).hexdigest(), + } + ) + identity = { + "schema_version": E39_PACKAGE_SCHEMA, + "classification": "immutable-ravnoves00-r1-camera-lidar-worker-input", + "source_ids": expected_ids, + "profile_sha256": _sha256(profile_source), + "runtime_requirements": { + "python": "3.12", + "numpy": "1.26+", + }, + "build_requirements": {"pillow": "10+"}, + "artifact_paths": [row["path"] for row in descriptors], + "source_artifacts": descriptors, + "authority": { + "commands_enabled": False, + "navigation_or_safety_accepted": False, + }, + } + identity_sha256 = hashlib.sha256(_canonical_json(identity)).hexdigest() + package_id = f"e39-worker-package-{identity_sha256}" + output = output_root.expanduser().absolute() + output.mkdir(mode=0o700, parents=True, exist_ok=True) + destination = output / package_id + if destination.exists(): + validate_e39_worker_package(destination) + return destination + + staging = output / f".{package_id}.{uuid.uuid4().hex}.tmp" + staging.mkdir(mode=0o700, exist_ok=False) + try: + for relative, source in sources.items(): + target = staging / relative + target.parent.mkdir(mode=0o700, parents=True, exist_ok=True) + if source is None: + target.write_text(_GENERATED_COMPUTE_INIT, encoding="utf-8") + elif isinstance(source, bytes): + target.write_bytes(source) + else: + shutil.copyfile(source, target) + artifacts = [ + { + "kind": relative, + "path": relative, + "byte_length": (staging / relative).stat().st_size, + "sha256": _sha256(staging / relative), + } + for relative in sorted(sources) + ] + manifest = { + "schema_version": E39_PACKAGE_SCHEMA, + "package_id": package_id, + "identity_sha256": identity_sha256, + "identity": identity, + "created_at_utc": datetime.now(UTC) + .isoformat(timespec="milliseconds") + .replace("+00:00", "Z"), + "artifacts": artifacts, + } + _write_json(staging / "manifest.json", manifest) + validate_e39_worker_package(staging, allow_staging=True) + os.replace(staging, destination) + except BaseException: + shutil.rmtree(staging, ignore_errors=True) + raise + validate_e39_worker_package(destination) + return destination + + +def validate_e39_worker_package( + root: Path, + *, + allow_staging: bool = False, +) -> dict[str, Any]: + """Validate package identity, exact file set, and every member digest.""" + + resolved = root.resolve(strict=True) + manifest = _read_json(resolved / "manifest.json") + identity = manifest.get("identity") + identity_sha256 = manifest.get("identity_sha256") + package_id = manifest.get("package_id") + artifacts = manifest.get("artifacts") + expected_name = ( + isinstance(package_id, str) + and ( + resolved.name == package_id + or ( + allow_staging + and resolved.name.startswith(f".{package_id}.") + and resolved.name.endswith(".tmp") + ) + ) + ) + if ( + manifest.get("schema_version") != E39_PACKAGE_SCHEMA + or not isinstance(identity, dict) + or not isinstance(identity_sha256, str) + or hashlib.sha256(_canonical_json(identity)).hexdigest() != identity_sha256 + or package_id != f"e39-worker-package-{identity_sha256}" + or not expected_name + or not isinstance(artifacts, list) + ): + raise E39WorkerPackageError("E39 worker package identity is invalid") + expected_paths = set(identity.get("artifact_paths", [])) + actual_paths = { + path.relative_to(resolved).as_posix() + for path in resolved.rglob("*") + if path.is_file() + } + if ( + not expected_paths + or actual_paths != expected_paths | {"manifest.json"} + or len(artifacts) != len(expected_paths) + ): + raise E39WorkerPackageError("E39 worker package file set changed") + observed: set[str] = set() + for row in artifacts: + if not isinstance(row, dict): + raise E39WorkerPackageError("E39 worker package artifact is invalid") + relative = row.get("path") + path = resolved / str(relative) + if ( + not isinstance(relative, str) + or relative not in expected_paths + or relative in observed + or Path(relative).is_absolute() + or ".." in Path(relative).parts + or not path.is_file() + or path.is_symlink() + or row.get("kind") != relative + or row.get("byte_length") != path.stat().st_size + or row.get("sha256") != _sha256(path) + ): + raise E39WorkerPackageError("E39 worker package artifact changed") + observed.add(relative) + if observed != expected_paths: + raise E39WorkerPackageError("E39 worker package coverage changed") + return manifest + + +def _add_materialized_evidence( + sources: dict[str, Path | bytes | None], + *, + materialization: Path, +) -> None: + rows = _read_jsonl(materialization / "materialized-items.jsonl") + if len(rows) != 486: + raise E39WorkerPackageError("E39 materialization denominator changed") + for row in rows: + for descriptor_name in ("artifact", "camera_frame"): + descriptor = row.get(descriptor_name) + if not isinstance(descriptor, dict): + raise E39WorkerPackageError( + "E39 materialization descriptor is invalid" + ) + relative = descriptor.get("path") + if ( + not isinstance(relative, str) + or Path(relative).is_absolute() + or ".." in Path(relative).parts + ): + raise E39WorkerPackageError("E39 materialization path is invalid") + source = materialization / relative + if ( + not source.is_file() + or source.is_symlink() + or descriptor.get("byte_length") != source.stat().st_size + or descriptor.get("sha256") != _sha256(source) + ): + raise E39WorkerPackageError( + "E39 materialized evidence content changed" + ) + target = ( + f"input/materialization/{materialization.name}/{relative}" + ) + existing = sources.get(target) + if existing is not None and existing != source: + raise E39WorkerPackageError("E39 package target collision") + sources[target] = source + + +def _add_feature_cache( + sources: dict[str, Path | bytes | None], + *, + acceptance: Path, + materialization: Path, +) -> None: + acceptance_rows = _read_jsonl(acceptance / "acceptance-items.jsonl") + materialization_rows = _read_jsonl( + materialization / "materialized-items.jsonl" + ) + acceptance_by_id = { + str(row["item_id"]): row for row in acceptance_rows + } + if ( + len(acceptance_by_id) != 486 + or {str(row["item_id"]) for row in materialization_rows} + != set(acceptance_by_id) + ): + raise E39WorkerPackageError("E39 feature-cache denominator changed") + item_ids = [str(row["item_id"]) for row in materialization_rows] + features = np.asarray( + [ + _feature_vector( + acceptance_by_id[item_id], + row, + materialization, + ) + for item_id, row in zip(item_ids, materialization_rows, strict=True) + ], + dtype=np.float64, + ) + names = _feature_names() + if features.shape != (486, len(names)) or not np.isfinite(features).all(): + raise E39WorkerPackageError("E39 feature-cache matrix is invalid") + arrays_stream = io.BytesIO() + np.savez_compressed( + arrays_stream, + item_ids=np.asarray(item_ids, dtype=f" bytes: + return json.dumps( + value, + sort_keys=True, + separators=(",", ":"), + allow_nan=False, + ).encode() + + +def _sha256(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as stream: + while chunk := stream.read(1024 * 1024): + digest.update(chunk) + return digest.hexdigest() + + +def _read_json(path: Path) -> dict[str, Any]: + value = json.loads(path.read_text(encoding="utf-8-sig")) + if not isinstance(value, dict): + raise E39WorkerPackageError(f"JSON object expected: {path.name}") + return value + + +def _read_jsonl(path: Path) -> list[dict[str, Any]]: + rows = [ + json.loads(line) + for line in path.read_text(encoding="utf-8-sig").splitlines() + ] + if not all(isinstance(row, dict) for row in rows): + raise E39WorkerPackageError(f"JSONL object expected: {path.name}") + return rows + + +def _write_json(path: Path, value: object) -> None: + with path.open("x", encoding="utf-8", newline="\n") as stream: + json.dump(value, stream, indent=2, sort_keys=True) + stream.write("\n") + stream.flush() + os.fsync(stream.fileno()) + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--repository-root", type=Path, required=True) + parser.add_argument("--acceptance", type=Path, required=True) + parser.add_argument("--materialization", type=Path, required=True) + parser.add_argument("--profile", type=Path, required=True) + parser.add_argument("--output-root", type=Path, required=True) + args = parser.parse_args() + package = build_e39_worker_package( + repository_root=args.repository_root, + acceptance_root=args.acceptance, + materialization_root=args.materialization, + profile_path=args.profile, + output_root=args.output_root, + ) + print(package) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/experiments/perception/run_e39_perception_refinement.py b/experiments/perception/run_e39_perception_refinement.py new file mode 100644 index 0000000..1f15861 --- /dev/null +++ b/experiments/perception/run_e39_perception_refinement.py @@ -0,0 +1,53 @@ +#!/usr/bin/env python3 +"""Build the development-qualified RAVNOVES00 R1 refinement.""" + +from __future__ import annotations + +import argparse +import json +import os +from pathlib import Path + +from k1link.compute.e39_perception_refinement import ( + build_e39_perception_refinement, +) + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--acceptance", type=Path, required=True) + parser.add_argument("--materialization", type=Path, required=True) + parser.add_argument("--profile", type=Path, required=True) + parser.add_argument("--output-root", type=Path, required=True) + parser.add_argument("--worker-node", default=os.environ.get("COMPUTERNAME")) + args = parser.parse_args() + result = build_e39_perception_refinement( + acceptance_root=args.acceptance, + materialization_root=args.materialization, + profile_path=args.profile, + output_root=args.output_root, + worker_node=args.worker_node, + ) + print( + json.dumps( + { + "result_id": result.result_id, + "result_root": str(result.result_root), + "quality_gate_passed": result.quality_gate_passed, + "development_cross_validation": result.report[ + "development_cross_validation" + ], + "metrics": result.report["metrics"], + "blocking_checks": result.report["quality_gate"][ + "blocking_checks" + ], + }, + ensure_ascii=False, + sort_keys=True, + ) + ) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/experiments/perception/worker/Invoke-E39PerceptionRefinement.ps1 b/experiments/perception/worker/Invoke-E39PerceptionRefinement.ps1 new file mode 100644 index 0000000..333a431 --- /dev/null +++ b/experiments/perception/worker/Invoke-E39PerceptionRefinement.ps1 @@ -0,0 +1,138 @@ +[CmdletBinding()] +param( + [Parameter(Mandatory = $true)] + [string]$PackageRoot, + [string]$OutputRoot = "D:\NDC_MISSIONCORE\runtime\derived\e39-refinement", + [string]$ContainerImage = "nvcr.io/nvidia/tritonserver:26.06-py3@sha256:58df7489c3f2276f9591d500a012dee03e23d35543ce3c390b4c001e6bf90794", + [ValidateRange(1, 1000)] + [int]$FreeGiBFloor = 300 +) + +$ErrorActionPreference = "Stop" +$ProgressPreference = "SilentlyContinue" + +function Assert-LastExitCode([string]$Operation) { + if ($LASTEXITCODE -ne 0) { + throw "$Operation failed with exit code $LASTEXITCODE" + } +} + +function Resolve-DDirectory([string]$Path, [string]$Label) { + $item = Get-Item -LiteralPath (Resolve-Path -LiteralPath $Path).Path -Force + $root = [IO.Path]::GetPathRoot($item.FullName).TrimEnd("\") + if ( + -not $item.PSIsContainer -or + ($item.Attributes -band [IO.FileAttributes]::ReparsePoint) -or + $root -ine "D:" + ) { + throw "$Label must be a real D: directory" + } + return $item.FullName +} + +function Convert-ToDockerPath([string]$Path) { + return $Path.Replace("\", "/") +} + +function Assert-FreeSpace([string]$Phase) { + $free = [int64](Get-PSDrive -Name D).Free + $floor = [int64]$FreeGiBFloor * 1GB + Write-Host ( + "DISK_GUARD PHASE={0} DRIVE=D FREE_BYTES={1} FREE_GIB={2} FLOOR_GIB={3}" -f + $Phase, $free, [math]::Round($free / 1GB, 3), $FreeGiBFloor + ) + if ($free -lt ($floor + 1GB)) { + throw "D: lacks the guarded E39 reserve during $Phase" + } + return $free +} + +$package = Resolve-DDirectory $PackageRoot "E39 package" +$packageManifestPath = Join-Path $package "manifest.json" +if (-not (Test-Path -LiteralPath $packageManifestPath -PathType Leaf)) { + throw "E39 package manifest is missing" +} +$packageManifest = Get-Content -LiteralPath $packageManifestPath -Raw | + ConvertFrom-Json +if ( + $packageManifest.schema_version -ne "missioncore.e39-worker-package/v1" -or + $packageManifest.package_id -ne (Split-Path $package -Leaf) -or + $packageManifest.package_id -notmatch "^e39-worker-package-[a-f0-9]{64}$" +) { + throw "E39 package manifest is incompatible" +} + +if (-not (Test-Path -LiteralPath $OutputRoot)) { + $null = New-Item -ItemType Directory -Path $OutputRoot +} +$output = Resolve-DDirectory $OutputRoot "E39 output root" +$freeBefore = Assert-FreeSpace "preflight" + +& docker image inspect $ContainerImage *> $null +Assert-LastExitCode "Pinned E39 container image inspection" + +$dockerPackage = Convert-ToDockerPath $package +$dockerOutput = Convert-ToDockerPath $output +$packageName = Split-Path $package -Leaf +$containerPackage = "/opt/e39-input/$packageName" +$command = @( + "run", "--rm", + "--name", "ndc-mission-core-e39-refinement", + "--network", "none", + "--read-only", + "--security-opt", "no-new-privileges:true", + "--cap-drop", "ALL", + "--pids-limit", "128", + "--memory", "1g", + "--memory-swap", "1g", + "--cpus", "2", + "--tmpfs", "/tmp:rw,noexec,nosuid,size=64m", + "-e", "PYTHONDONTWRITEBYTECODE=1", + "-e", ("PYTHONPATH={0}/runtime" -f $containerPackage), + "-e", ("E39_WORKER_NODE={0}" -f $env:COMPUTERNAME), + "-v", ("{0}:{1}:ro" -f $dockerPackage, $containerPackage), + "-v", ("{0}:/output:rw" -f $dockerOutput), + "--entrypoint", "python3", + $ContainerImage, + ("{0}/runtime/run_e39_perception_refinement.py" -f $containerPackage), + "--package", $containerPackage, + "--output-root", "/output" +) + +Write-Output ("PACKAGE_ID={0}" -f $packageManifest.package_id) +Write-Output ("PACKAGE_IDENTITY_SHA256={0}" -f $packageManifest.identity_sha256) +Write-Output ("CONTAINER_IMAGE={0}" -f $ContainerImage) +& docker @command +Assert-LastExitCode "E39 perception refinement" + +$matches = @( + Get-ChildItem -LiteralPath $output -Directory -Filter "e39-perception-refinement-*" | + Where-Object { + $manifestPath = Join-Path $_.FullName "manifest.json" + if (-not (Test-Path -LiteralPath $manifestPath -PathType Leaf)) { + return $false + } + $manifest = Get-Content -LiteralPath $manifestPath -Raw | + ConvertFrom-Json + return ( + $manifest.schema_version -eq + "missioncore.e39-perception-refinement/v1" -and + $manifest.acceptance_state -eq + "completed-r1-source-scoped-refinement" -and + $manifest.identity.execution.worker_node -eq $env:COMPUTERNAME + ) + } +) +if ($matches.Count -ne 1) { + throw "E39 immutable result could not be resolved uniquely" +} +$resultRoot = $matches[0].FullName +$resultManifest = Get-Content -LiteralPath ( + Join-Path $resultRoot "manifest.json" +) -Raw | ConvertFrom-Json +$freeAfter = Assert-FreeSpace "completed" +Write-Output ("RESULT_ROOT={0}" -f $resultRoot) +Write-Output ("RESULT_ID={0}" -f $resultManifest.result_id) +Write-Output ("QUALITY_GATE_PASSED={0}" -f $resultManifest.quality_gate_passed) +Write-Output ("DISK_FREE_BYTES_BEFORE={0}" -f $freeBefore) +Write-Output ("DISK_FREE_BYTES_AFTER={0}" -f $freeAfter) diff --git a/experiments/perception/worker/run_e39_perception_refinement.py b/experiments/perception/worker/run_e39_perception_refinement.py new file mode 100644 index 0000000..af7e5cc --- /dev/null +++ b/experiments/perception/worker/run_e39_perception_refinement.py @@ -0,0 +1,53 @@ +#!/usr/bin/env python3 +"""Execute a packaged E39 refinement inside the pinned Worker 006 container.""" + +from __future__ import annotations + +import argparse +import json +import os +from pathlib import Path + +from k1link.compute.e39_perception_refinement import ( + build_e39_perception_refinement, +) + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--package", type=Path, required=True) + parser.add_argument("--output-root", type=Path, required=True) + args = parser.parse_args() + package = args.package.resolve(strict=True) + profile = json.loads((package / "profile.json").read_text(encoding="utf-8")) + acceptance_id = profile["source"]["acceptance_result_id"] + materialization_id = profile["source"]["materialization_id"] + result = build_e39_perception_refinement( + acceptance_root=package / "input" / "acceptance" / acceptance_id, + materialization_root=( + package / "input" / "materialization" / materialization_id + ), + profile_path=package / "profile.json", + output_root=args.output_root, + worker_node=os.environ.get("E39_WORKER_NODE"), + ) + print( + json.dumps( + { + "result_id": result.result_id, + "result_root": str(result.result_root), + "quality_gate_passed": result.quality_gate_passed, + "metrics": result.report["metrics"], + "blocking_checks": result.report["quality_gate"][ + "blocking_checks" + ], + }, + ensure_ascii=False, + sort_keys=True, + ) + ) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/k1link/compute/e39_perception_refinement.py b/src/k1link/compute/e39_perception_refinement.py new file mode 100644 index 0000000..54019c6 --- /dev/null +++ b/src/k1link/compute/e39_perception_refinement.py @@ -0,0 +1,1149 @@ +"""Development-qualified RAVNOVES00 R1 perception refinement. + +E39 keeps the E37 denominator and validation split immutable. It enriches the +E38 tabular baseline with source-scoped camera and LiDAR shape features, chooses +the fixed model contract through development-only cross-validation, fits only +on development labels, and evaluates the sealed validation partition once. + +The result is diagnostic. It grants no navigation, command, or safety +authority and makes no claim about another route, rig, camera, or mount. +""" + +from __future__ import annotations + +import hashlib +import json +import math +import os +import shutil +import uuid +from collections import Counter, defaultdict +from dataclasses import dataclass +from datetime import UTC, datetime +from pathlib import Path +from typing import Any, Final + +import numpy as np + +from k1link.compute.e37_acceptance_contract import ( + E37_ITEMS_NAME, + E37AcceptanceContractError, + read_e37_acceptance_contract, +) + +E39_PROFILE_SCHEMA: Final = "missioncore.e39-perception-refinement-profile/v1" +E39_PACKAGE_SCHEMA: Final = "missioncore.e39-worker-package/v1" +E39_RESULT_SCHEMA: Final = "missioncore.e39-perception-refinement/v1" +E39_PREDICTION_SCHEMA: Final = "missioncore.e39-perception-prediction/v1" +E39_MODEL_SCHEMA: Final = "missioncore.e39-development-knn-model/v1" +E39_REPORT_SCHEMA: Final = "missioncore.e39-perception-refinement-report/v1" + +E39_PREDICTIONS_NAME: Final = "predictions.jsonl" +E39_MODEL_NAME: Final = "development-model.json" +E39_REPORT_NAME: Final = "run-report.json" +E39_MANIFEST_NAME: Final = "manifest.json" + +_MATERIALIZATION_SCHEMA: Final = "missioncore.e30-evidence-materialization/v2" +_FEATURE_CACHE_SCHEMA: Final = "missioncore.e39-feature-cache/v1" +_FEATURE_CACHE_MANIFEST: Final = "e39-feature-cache.json" +_FEATURE_CACHE_ARRAYS: Final = "e39-feature-vectors.npz" +_SOURCE_SESSION_ID: Final = "20260720T065719Z_viewer_live" +_SOURCE_DISPLAY_NAME: Final = "RAVNOVES00" +_DIMENSIONS: Final = ("presence", "geometry_association", "freshness") +_STRATA: Final = ("agree", "camera-only", "conflict", "geometry-only", "unknown") +_RANGES: Final = ("near", "middle", "far", "unavailable") +_GEOMETRIES: Final = ( + "agree", + "conflict", + "single-source-camera", + "single-source-geometry", + "unknown", + "unavailable", +) +_QUANTILES: Final = ("min", "p10", "p25", "p50", "p75", "p90", "max") +_AUTHORITY: Final = { + "commands_enabled": False, + "navigation_or_safety_accepted": False, +} + + +class E39PerceptionRefinementError(RuntimeError): + """The E39 profile, immutable inputs, model, or result is invalid.""" + + +@dataclass(frozen=True, slots=True) +class E39PerceptionRefinement: + result_id: str + result_root: Path + manifest: dict[str, Any] + report: dict[str, Any] + model: dict[str, Any] + + @property + def quality_gate_passed(self) -> bool: + return self.report.get("quality_gate", {}).get("passed") is True + + +def build_e39_perception_refinement( + *, + acceptance_root: Path, + materialization_root: Path, + profile_path: Path, + output_root: Path, + worker_node: str | None = None, +) -> E39PerceptionRefinement: + """Fit the frozen E39 contract and evaluate the sealed validation split.""" + + profile_file = profile_path.resolve(strict=True) + profile = _read_json(profile_file) + _validate_profile(profile) + acceptance_path = acceptance_root.resolve(strict=True) + materialization_path = materialization_root.resolve(strict=True) + try: + acceptance = read_e37_acceptance_contract(acceptance_path) + except E37AcceptanceContractError as exc: + raise E39PerceptionRefinementError("E39 E37 contract is invalid") from exc + if acceptance.result_id != profile["source"]["acceptance_result_id"]: + raise E39PerceptionRefinementError("E39 acceptance identity changed") + + acceptance_items_path = acceptance_path / E37_ITEMS_NAME + acceptance_rows = _read_jsonl(acceptance_items_path) + materialization_rows, materialization_binding = _load_materialization( + materialization_path, + acceptance.manifest, + profile, + ) + feature_cache, feature_cache_binding = _load_feature_cache( + materialization_path, + materialization_binding, + ) + if ( + len(acceptance_rows) != 486 + or len(materialization_rows) != 486 + or {row.get("item_id") for row in acceptance_rows} + != {row.get("item_id") for row in materialization_rows} + or ( + feature_cache is not None + and set(feature_cache) + != {str(row.get("item_id")) for row in materialization_rows} + ) + ): + raise E39PerceptionRefinementError("E39 denominator accounting differs") + + identity = { + "schema_version": E39_RESULT_SCHEMA, + "source": { + "session_id": _SOURCE_SESSION_ID, + "display_name": _SOURCE_DISPLAY_NAME, + "acceptance_result_id": acceptance.result_id, + "acceptance_identity_sha256": acceptance.manifest["identity_sha256"], + "acceptance_items_sha256": _sha256(acceptance_items_path), + **materialization_binding, + "feature_projection": feature_cache_binding, + }, + "profile": { + "profile_id": profile["profile_id"], + "sha256": _sha256(profile_file), + }, + "execution": { + "class": "development-qualified-sealed-validation-evaluation", + "worker_node": worker_node or "unbound-local", + }, + "authority": _AUTHORITY, + } + identity_sha256 = hashlib.sha256(_canonical_json(identity)).hexdigest() + result_id = f"e39-perception-refinement-{identity_sha256}" + destination = output_root.expanduser().absolute() / result_id + if destination.exists(): + return read_e39_perception_refinement(destination) + + materialization_by_id = { + str(row["item_id"]): row for row in materialization_rows + } + joined: list[dict[str, Any]] = [] + for acceptance_row in acceptance_rows: + item_id = str(acceptance_row.get("item_id")) + reference = acceptance_row.get("reference") + if ( + acceptance_row.get("split") not in {"development", "validation"} + or acceptance_row.get("severity") not in {"standard", "medium", "high"} + or not isinstance(reference, dict) + or set(reference) != set(_DIMENSIONS) + ): + raise E39PerceptionRefinementError("E39 acceptance row is invalid") + joined.append( + { + "acceptance": acceptance_row, + "features": _feature_vector( + acceptance_row, + materialization_by_id[item_id], + materialization_path, + ) + if feature_cache is None + else feature_cache[item_id], + } + ) + + feature_names = _feature_names() + if any(len(row["features"]) != len(feature_names) for row in joined): + raise E39PerceptionRefinementError("E39 feature contract changed") + development = [ + row for row in joined if row["acceptance"]["split"] == "development" + ] + validation = [ + row for row in joined if row["acceptance"]["split"] == "validation" + ] + if len(development) != 340 or len(validation) != 146: + raise E39PerceptionRefinementError("E39 frozen split changed") + + model_profile = profile["model"] + neighbors = int(model_profile["neighbors"]) + clip = float(model_profile["robust_clip"]) + development_matrix = np.asarray( + [row["features"] for row in development], + dtype=np.float64, + ) + median, scale = _fit_robust_scaler(development_matrix) + transformed_development = _transform( + development_matrix, + median=median, + scale=scale, + clip=clip, + ) + development_labels = [ + str(row["acceptance"]["reference"]["presence"]) + for row in development + ] + cross_validation = _development_cross_validation( + development, + neighbors=neighbors, + clip=clip, + seed=str(model_profile["cross_validation_seed"]), + folds=int(model_profile["cross_validation_folds"]), + targets=profile["targets"], + ) + + predictions: list[dict[str, Any]] = [] + for row in joined: + transformed = _transform( + np.asarray([row["features"]], dtype=np.float64), + median=median, + scale=scale, + clip=clip, + )[0] + presence, confidence = _predict_presence( + transformed, + transformed_development, + development_labels, + neighbors=neighbors, + ) + stratum = str(row["acceptance"]["source_stratum"]) + predicted = _project_dimensions(stratum, presence) + acceptance_row = row["acceptance"] + predictions.append( + { + "schema_version": E39_PREDICTION_SCHEMA, + "sequence": acceptance_row["sequence"], + "item_id": acceptance_row["item_id"], + "review_key": acceptance_row["review_key"], + "source_frame_index": acceptance_row["source_frame_index"], + "source_stratum": stratum, + "severity": acceptance_row["severity"], + "split": acceptance_row["split"], + "prediction": predicted, + "presence_vote_confidence": round(confidence, 6), + "reference": acceptance_row["reference"], + "scored": acceptance_row["split"] == "validation", + "authority": _AUTHORITY, + } + ) + + validation_predictions = [ + row for row in predictions if row["split"] == "validation" + ] + dimension_metrics = { + dimension: _dimension_metrics( + validation_predictions, + dimension=dimension, + target=float(profile["targets"][f"{dimension}_target"]), + ) + for dimension in _DIMENSIONS + } + high_severity_failures = sum( + row["severity"] == "high" + and any( + row["prediction"][dimension] != row["reference"][dimension] + for dimension in _DIMENSIONS + ) + for row in validation_predictions + ) + accounting_fraction = len(validation_predictions) / 146 + false_free_claims = sum( + value == "free" + for row in predictions + for value in row["prediction"].values() + ) + gate_checks = { + "development_cv_targets_reached": cross_validation["passed"], + "presence_target_reached": dimension_metrics["presence"]["passed"], + "geometry_association_target_reached": dimension_metrics[ + "geometry_association" + ]["passed"], + "freshness_target_reached": dimension_metrics["freshness"]["passed"], + "validation_accounting_complete": math.isclose(accounting_fraction, 1.0), + "false_free_claims_zero": false_free_claims == 0, + "high_severity_failures_zero": high_severity_failures == 0, + "authority_remains_diagnostic": True, + } + quality_gate_passed = all(gate_checks.values()) + model_document = { + "schema_version": E39_MODEL_SCHEMA, + "profile_id": profile["profile_id"], + "feature_set": model_profile["feature_set"], + "feature_names": feature_names, + "training_split": "development", + "training_items": len(development), + "validation_labels_used_for_training": False, + "neighbors": neighbors, + "robust_clip": clip, + "scaler": { + "median": _rounded_vector(median), + "scale": _rounded_vector(scale), + }, + "training_rows": [ + { + "item_id": row["acceptance"]["item_id"], + "presence": label, + "vector": _rounded_vector(vector), + } + for row, label, vector in zip( + development, + development_labels, + transformed_development, + strict=True, + ) + ], + "dimension_projection": "source-stratum-plus-presence/v1", + "development_cross_validation": cross_validation, + "authority": _AUTHORITY, + } + report = { + "schema_version": E39_REPORT_SCHEMA, + "result_id": result_id, + "identity_sha256": identity_sha256, + "status": "measured-r1-source-scoped-refinement", + "source_session_id": _SOURCE_SESSION_ID, + "source_display_name": _SOURCE_DISPLAY_NAME, + "profile_id": profile["profile_id"], + "execution": identity["execution"], + "method": { + "summary": ( + "robust-scaled 3-neighbour presence classification over exact " + "camera crops and LiDAR shape/projection statistics" + ), + "selection": ( + "the fixed profile is qualified only through deterministic " + "five-fold development cross-validation" + ), + "dimension_projection": ( + "geometry association and freshness are derived from the " + "predicted presence state plus the immutable source stratum" + ), + }, + "development_cross_validation": cross_validation, + "metrics": { + "development_items": len(development), + "validation_items": len(validation_predictions), + "terminal_outcomes": len(validation_predictions), + "accounting_fraction": round(accounting_fraction, 6), + "false_free_claims": false_free_claims, + "high_severity_failures": high_severity_failures, + "dimensions": dimension_metrics, + }, + "quality_gate": { + "passed": quality_gate_passed, + "checks": gate_checks, + "blocking_checks": [ + name for name, passed in gate_checks.items() if not passed + ], + }, + "decision": { + "r1_refinement_measured": True, + "accepted_for_release": quality_gate_passed, + "next_gate": ( + "R2 source-scoped temporal product state" + if quality_gate_passed + else "R1 remaining high-severity and dimension errors" + ), + }, + "limitations": [ + ( + "the model is source-scoped to RAVNOVES00 and includes source " + "frame coordinates; it is not a route-independent detector" + ), + ( + "labels are an engineering-reviewed substrate rather than " + "independent physical ground truth" + ), + "validation labels are evaluation-only and never fit the model", + ( + "the result does not prove another route, camera, rig, mount, " + "vegetation environment, or rover installation" + ), + "navigation, command and safety authority remain false", + ], + "authority": _AUTHORITY, + } + + destination.parent.mkdir(mode=0o700, parents=True, exist_ok=True) + staging = destination.parent / f".{result_id}.{uuid.uuid4().hex}.tmp" + staging.mkdir(mode=0o700, exist_ok=False) + try: + _write_jsonl(staging / E39_PREDICTIONS_NAME, predictions) + _write_json(staging / E39_MODEL_NAME, model_document) + _write_json(staging / E39_REPORT_NAME, report) + artifacts = [ + _artifact(staging / E39_PREDICTIONS_NAME, "sealed-evaluation"), + _artifact(staging / E39_MODEL_NAME, "development-qualified-model"), + _artifact(staging / E39_REPORT_NAME, "quality-report"), + ] + manifest = { + "schema_version": E39_RESULT_SCHEMA, + "result_id": result_id, + "identity_sha256": identity_sha256, + "identity": identity, + "created_at_utc": _utc_now(), + "acceptance_state": "completed-r1-source-scoped-refinement", + "quality_gate_passed": quality_gate_passed, + "artifacts": artifacts, + } + _write_json(staging / E39_MANIFEST_NAME, manifest) + os.replace(staging, destination) + except BaseException: + shutil.rmtree(staging, ignore_errors=True) + raise + return read_e39_perception_refinement(destination) + + +def read_e39_perception_refinement(root: Path) -> E39PerceptionRefinement: + """Validate and read one immutable E39 result.""" + + resolved = root.resolve(strict=True) + manifest = _read_json(resolved / E39_MANIFEST_NAME) + identity = manifest.get("identity") + identity_sha256 = manifest.get("identity_sha256") + if ( + manifest.get("schema_version") != E39_RESULT_SCHEMA + or not isinstance(identity, dict) + or not isinstance(identity_sha256, str) + or hashlib.sha256(_canonical_json(identity)).hexdigest() != identity_sha256 + or manifest.get("result_id") != f"e39-perception-refinement-{identity_sha256}" + or resolved.name != manifest.get("result_id") + or manifest.get("acceptance_state") + != "completed-r1-source-scoped-refinement" + or identity.get("authority") != _AUTHORITY + ): + raise E39PerceptionRefinementError("E39 result identity is invalid") + expected = { + E39_PREDICTIONS_NAME: "sealed-evaluation", + E39_MODEL_NAME: "development-qualified-model", + E39_REPORT_NAME: "quality-report", + } + artifacts = manifest.get("artifacts") + if not isinstance(artifacts, list) or len(artifacts) != len(expected): + raise E39PerceptionRefinementError("E39 artifact catalog is invalid") + for row in artifacts: + if not isinstance(row, dict): + raise E39PerceptionRefinementError("E39 artifact descriptor is invalid") + name = row.get("path") + path = resolved / str(name) + if ( + name not in expected + or row.get("role") != expected[name] + or not path.is_file() + or path.is_symlink() + or row.get("byte_length") != path.stat().st_size + or row.get("sha256") != _sha256(path) + ): + raise E39PerceptionRefinementError("E39 artifact content changed") + report = _read_json(resolved / E39_REPORT_NAME) + model = _read_json(resolved / E39_MODEL_NAME) + if ( + report.get("schema_version") != E39_REPORT_SCHEMA + or report.get("result_id") != resolved.name + or report.get("identity_sha256") != identity_sha256 + or report.get("status") != "measured-r1-source-scoped-refinement" + or model.get("schema_version") != E39_MODEL_SCHEMA + or model.get("validation_labels_used_for_training") is not False + or manifest.get("quality_gate_passed") + is not report.get("quality_gate", {}).get("passed") + ): + raise E39PerceptionRefinementError("E39 report or model is invalid") + return E39PerceptionRefinement( + result_id=resolved.name, + result_root=resolved, + manifest=manifest, + report=report, + model=model, + ) + + +def _load_materialization( + root: Path, + acceptance_manifest: dict[str, Any], + profile: dict[str, Any], +) -> tuple[list[dict[str, Any]], dict[str, Any]]: + manifest_path = root / "manifest.json" + index_path = root / "materialized-items.jsonl" + manifest = _read_json(manifest_path) + binding = acceptance_manifest.get("identity", {}).get("reviewed_substrate", {}) + if ( + manifest.get("schema_version") != _MATERIALIZATION_SCHEMA + or root.name != profile["source"]["materialization_id"] + or manifest.get("result_id") != root.name + or root.name != binding.get("materialization_id") + or manifest.get("identity_sha256") + != binding.get("materialization_identity_sha256") + or _sha256(manifest_path) + != binding.get("materialization_manifest_sha256") + or not index_path.is_file() + or index_path.is_symlink() + or _sha256(index_path) != binding.get("materialization_index_sha256") + ): + raise E39PerceptionRefinementError("E39 materialization identity changed") + rows = _read_jsonl(index_path) + return rows, { + "materialization_id": root.name, + "materialization_identity_sha256": manifest["identity_sha256"], + "materialization_index_sha256": _sha256(index_path), + } + + +def _feature_names() -> list[str]: + names = [ + "detector_score", + "selected_points", + "candidate_points", + "rejected_points", + "projected_points", + "frame_points", + "nearest_range_m", + "point_count", + "voxel_count", + "bounds_span_x_m", + "bounds_span_y_m", + "bounds_span_z_m", + "source_frame_index_scaled", + "review_ordinal", + ] + names += [f"stratum={value}" for value in _STRATA] + names += [f"range={value}" for value in _RANGES] + names += [f"geometry={value}" for value in _GEOMETRIES] + for prefix in ("candidate_xyz", "selected_xyz"): + for axis in ("x", "y", "z"): + names += [f"{prefix}_{axis}_{value}" for value in _QUANTILES] + names += [f"{prefix}_covariance_ratio_{index}" for index in range(3)] + for prefix in ("candidate_pixels", "selected_pixels"): + for axis in ("x", "y"): + names += [f"{prefix}_{axis}_{value}" for value in _QUANTILES] + names += [ + f"{prefix}_span_x", + f"{prefix}_span_y", + f"{prefix}_density", + ] + for prefix in ( + "candidate_depth", + "selected_depth", + "candidate_height", + "selected_height", + ): + names += [f"{prefix}_{value}" for value in _QUANTILES] + names += [f"candidate_point_class_fraction_{index}" for index in range(8)] + names += [ + f"camera_crop_{channel}_{y}_{x}" + for y in range(6) + for x in range(6) + for channel in ("r", "g", "b") + ] + names += [f"camera_crop_luma_{value}" for value in _QUANTILES] + return names + + +def _load_feature_cache( + materialization_root: Path, + materialization_binding: dict[str, Any], +) -> tuple[dict[str, list[float]] | None, dict[str, Any]]: + manifest_path = materialization_root / _FEATURE_CACHE_MANIFEST + arrays_path = materialization_root / _FEATURE_CACHE_ARRAYS + if not manifest_path.exists() and not arrays_path.exists(): + return None, {"mode": "runtime-source-extraction"} + if ( + not manifest_path.is_file() + or manifest_path.is_symlink() + or not arrays_path.is_file() + or arrays_path.is_symlink() + ): + raise E39PerceptionRefinementError("E39 feature cache is incomplete") + manifest = _read_json(manifest_path) + names = _feature_names() + names_sha256 = hashlib.sha256(_canonical_json(names)).hexdigest() + if ( + manifest.get("schema_version") != _FEATURE_CACHE_SCHEMA + or manifest.get("materialization_id") != materialization_root.name + or manifest.get("materialization_index_sha256") + != materialization_binding["materialization_index_sha256"] + or manifest.get("feature_names_sha256") != names_sha256 + or manifest.get("items") != 486 + or manifest.get("dimensions") != len(names) + or manifest.get("arrays_path") != _FEATURE_CACHE_ARRAYS + or manifest.get("arrays_byte_length") != arrays_path.stat().st_size + or manifest.get("arrays_sha256") != _sha256(arrays_path) + ): + raise E39PerceptionRefinementError("E39 feature cache binding changed") + with np.load(arrays_path, allow_pickle=False) as arrays: + item_ids = arrays["item_ids"] + features = arrays["features"] + if ( + item_ids.shape != (486,) + or features.shape != (486, len(names)) + or not np.issubdtype(item_ids.dtype, np.str_) + or not np.isfinite(features).all() + or len(set(str(value) for value in item_ids)) != 486 + ): + raise E39PerceptionRefinementError("E39 feature cache arrays are invalid") + cache = { + str(item_id): [float(value) for value in vector] + for item_id, vector in zip(item_ids, features, strict=True) + } + return cache, { + "mode": "package-bound-feature-cache", + "schema_version": _FEATURE_CACHE_SCHEMA, + "manifest_sha256": _sha256(manifest_path), + "arrays_sha256": _sha256(arrays_path), + "feature_names_sha256": names_sha256, + } + + +def _feature_vector( + acceptance: dict[str, Any], + materialization: dict[str, Any], + materialization_root: Path, +) -> list[float]: + snapshot = _object(materialization.get("e29_snapshot"), "E39 snapshot") + evidence = _object( + materialization.get("materialization"), + "E39 materialization evidence", + ) + bounds = snapshot.get("bounds_map_xyz_m") + ordinal = str(acceptance.get("review_key", "")).rsplit(":", 1)[-1] + values = [ + _number_or(evidence.get("detector_score"), -1.0), + _number_or(evidence.get("selected_point_count"), 0.0), + _number_or(evidence.get("candidate_point_count"), 0.0), + _number_or(evidence.get("rejected_candidate_point_count"), 0.0), + _number_or(evidence.get("projected_point_count"), 0.0), + _number_or(evidence.get("frame_point_count"), 0.0), + _number_or(snapshot.get("nearest_range_m"), -1.0), + _number_or(snapshot.get("point_count"), 0.0), + _number_or(snapshot.get("voxel_count"), 0.0), + _span(bounds, 0), + _span(bounds, 1), + _span(bounds, 2), + _number_or(acceptance.get("source_frame_index"), -1.0) / 100.0, + _number_or(ordinal, -1.0), + ] + stratum = str(materialization.get("stratum")) + range_bucket = str(materialization.get("range_bucket")) + geometry = str(snapshot.get("geometry_status")) + values += [10.0 * float(stratum == item) for item in _STRATA] + values += [3.0 * float(range_bucket == item) for item in _RANGES] + values += [3.0 * float(geometry == item) for item in _GEOMETRIES] + + artifact = _object(materialization.get("artifact"), "E39 NPZ artifact") + artifact_path = materialization_root / str(artifact.get("path")) + if not artifact_path.is_file() or artifact_path.is_symlink(): + raise E39PerceptionRefinementError("E39 NPZ artifact is unavailable") + with np.load(artifact_path, allow_pickle=False) as arrays: + projected_pixels = arrays["projected_pixels_xy"] + candidate_mask = arrays["projected_candidate_mask"].astype(bool) + selected_mask = arrays["projected_selected_mask"].astype(bool) + values += _point_statistics(arrays["candidate_points_map_xyz_m"]) + values += _point_statistics(arrays["selected_points_map_xyz_m"]) + width = max(1, int(evidence.get("projection_width", 800))) + height = max(1, int(evidence.get("projection_height", 600))) + values += _pixel_statistics( + projected_pixels[candidate_mask], + width=width, + height=height, + ) + values += _pixel_statistics( + projected_pixels[selected_mask], + width=width, + height=height, + ) + values += _quantile_statistics( + arrays["projected_depth_m"][candidate_mask] + ) + values += _quantile_statistics(arrays["projected_depth_m"][selected_mask]) + values += _quantile_statistics( + arrays["projected_point_height_m"][candidate_mask] + ) + values += _quantile_statistics( + arrays["projected_point_height_m"][selected_mask] + ) + point_classes = arrays["projected_point_class"][candidate_mask] + values += [ + float(np.mean(point_classes == index)) if point_classes.size else 0.0 + for index in range(8) + ] + values += _camera_crop_features( + materialization, + materialization_root, + projected_pixels[candidate_mask], + width=width, + height=height, + ) + if not np.isfinite(np.asarray(values, dtype=np.float64)).all(): + raise E39PerceptionRefinementError("E39 produced a non-finite feature") + return values + + +def _point_statistics(points: np.ndarray[Any, Any]) -> list[float]: + values = np.asarray(points, dtype=np.float64) + if values.ndim != 2 or values.shape[1] != 3 or not len(values): + return [-1.0] * 24 + result: list[float] = [] + for axis in range(3): + result += _quantile_statistics(values[:, axis]) + if len(values) >= 3: + eigenvalues = np.maximum( + np.linalg.eigvalsh(np.cov(values, rowvar=False)), + 0.0, + ) + ratios = eigenvalues / (float(eigenvalues.sum()) + 1e-9) + result += [float(value) for value in ratios] + else: + result += [0.0, 0.0, 0.0] + return result + + +def _pixel_statistics( + pixels: np.ndarray[Any, Any], + *, + width: int, + height: int, +) -> list[float]: + values = np.asarray(pixels, dtype=np.float64) + if values.ndim != 2 or values.shape[1] != 2 or not len(values): + return [-1.0] * 17 + x = values[:, 0] / width + y = values[:, 1] / height + span_x = max(float(np.max(x) - np.min(x)), 1.0 / width) + span_y = max(float(np.max(y) - np.min(y)), 1.0 / height) + density = len(values) / (span_x * span_y * width * height + 1.0) + return ( + _quantile_statistics(x) + + _quantile_statistics(y) + + [span_x, span_y, density] + ) + + +def _camera_crop_features( + materialization: dict[str, Any], + materialization_root: Path, + candidate_pixels: np.ndarray[Any, Any], + *, + width: int, + height: int, +) -> list[float]: + from PIL import Image + + values = np.asarray(candidate_pixels, dtype=np.float64) + if values.ndim != 2 or values.shape[1] != 2 or not len(values): + return [0.0] * 115 + frame = _object(materialization.get("camera_frame"), "E39 camera frame") + frame_path = materialization_root / str(frame.get("path")) + if not frame_path.is_file() or frame_path.is_symlink(): + raise E39PerceptionRefinementError("E39 camera frame is unavailable") + low = np.min(values, axis=0) + high = np.max(values, axis=0) + center = (low + high) / 2.0 + support = np.maximum(high - low, 48.0) + box = ( + max(0, int(center[0] - support[0])), + max(0, int(center[1] - support[1])), + min(width, int(center[0] + support[0])), + min(height, int(center[1] + support[1])), + ) + if box[2] <= box[0] or box[3] <= box[1]: + return [0.0] * 115 + with Image.open(frame_path) as source: + crop = ( + source.convert("RGB") + .crop(box) + .resize((6, 6), resample=Image.Resampling.BILINEAR) + ) + rgb = np.asarray(crop, dtype=np.float64) / 255.0 + return [float(value) for value in rgb.reshape(-1)] + _quantile_statistics( + np.mean(rgb, axis=2).reshape(-1) + ) + + +def _quantile_statistics(values: object) -> list[float]: + array = np.asarray(values, dtype=np.float64).reshape(-1) + finite = array[np.isfinite(array)] + if not finite.size: + return [-1.0] * 7 + return [ + float(value) + for value in np.quantile(finite, (0.0, 0.1, 0.25, 0.5, 0.75, 0.9, 1.0)) + ] + + +def _fit_robust_scaler(matrix: np.ndarray[Any, Any]) -> tuple[np.ndarray, np.ndarray]: + median = np.median(matrix, axis=0) + scale = np.percentile(matrix, 75, axis=0) - np.percentile(matrix, 25, axis=0) + scale[scale < 1e-8] = 1.0 + return median, scale + + +def _transform( + matrix: np.ndarray[Any, Any], + *, + median: np.ndarray[Any, Any], + scale: np.ndarray[Any, Any], + clip: float, +) -> np.ndarray[Any, Any]: + return np.clip((matrix - median) / scale, -clip, clip) + + +def _predict_presence( + vector: np.ndarray[Any, Any], + training: np.ndarray[Any, Any], + labels: list[str], + *, + neighbors: int, +) -> tuple[str, float]: + distances = np.mean(np.square(training - vector), axis=1) + nearest = np.argsort(distances, kind="stable")[:neighbors] + votes = Counter(labels[int(index)] for index in nearest) + prediction, count = sorted( + votes.items(), + key=lambda item: (-item[1], item[0]), + )[0] + return prediction, count / neighbors + + +def _project_dimensions(stratum: str, presence: str) -> dict[str, str]: + if presence == "background-or-noise" or stratum == "conflict": + geometry = "rejected-nonobject" + elif presence == "unknown": + geometry = "unknown" + elif stratum == "camera-only": + geometry = "insufficient-support" + elif stratum == "unknown": + geometry = "unknown" + elif stratum == "geometry-only": + geometry = ( + "object-associated" + if presence == "object-present" + else "independent-occupied" + ) + else: + geometry = "object-associated" + + if stratum == "camera-only": + freshness = "current" if presence == "background-or-noise" else "unavailable" + elif stratum == "unknown": + freshness = "current" if presence == "background-or-noise" else "stale" + else: + freshness = "current" + return { + "presence": presence, + "geometry_association": geometry, + "freshness": freshness, + } + + +def _development_cross_validation( + development: list[dict[str, Any]], + *, + neighbors: int, + clip: float, + seed: str, + folds: int, + targets: dict[str, Any], +) -> dict[str, Any]: + assignments = np.asarray( + [ + int( + hashlib.sha256( + f"{seed}:{row['acceptance']['item_id']}".encode() + ).hexdigest()[:8], + 16, + ) + % folds + for row in development + ], + dtype=np.int64, + ) + matrix = np.asarray( + [row["features"] for row in development], + dtype=np.float64, + ) + predictions: list[dict[str, str] | None] = [None] * len(development) + for fold in range(folds): + train_indices = np.flatnonzero(assignments != fold) + test_indices = np.flatnonzero(assignments == fold) + median, scale = _fit_robust_scaler(matrix[train_indices]) + train_matrix = _transform( + matrix[train_indices], + median=median, + scale=scale, + clip=clip, + ) + test_matrix = _transform( + matrix[test_indices], + median=median, + scale=scale, + clip=clip, + ) + labels = [ + str(development[int(index)]["acceptance"]["reference"]["presence"]) + for index in train_indices + ] + for local_index, source_index in enumerate(test_indices): + presence, _ = _predict_presence( + test_matrix[local_index], + train_matrix, + labels, + neighbors=neighbors, + ) + predictions[int(source_index)] = _project_dimensions( + str( + development[int(source_index)]["acceptance"][ + "source_stratum" + ] + ), + presence, + ) + if any(value is None for value in predictions): + raise E39PerceptionRefinementError("E39 development CV is incomplete") + dimensions = {} + for dimension in _DIMENSIONS: + correct = sum( + prediction is not None + and prediction[dimension] + == row["acceptance"]["reference"][dimension] + for prediction, row in zip(predictions, development, strict=True) + ) + accuracy = correct / len(development) + target = float(targets[f"{dimension}_target"]) + dimensions[dimension] = { + "correct": correct, + "incorrect": len(development) - correct, + "total": len(development), + "accuracy": round(accuracy, 6), + "target": target, + "passed": accuracy >= target, + } + return { + "strategy": "deterministic-item-hash-five-fold", + "seed": seed, + "folds": folds, + "items": len(development), + "validation_labels_used": False, + "dimensions": dimensions, + "passed": all(row["passed"] for row in dimensions.values()), + } + + +def _dimension_metrics( + rows: list[dict[str, Any]], + *, + dimension: str, + target: float, +) -> dict[str, Any]: + confusion: Counter[tuple[str, str]] = Counter() + stratum_counts: dict[str, Counter[str]] = defaultdict(Counter) + correct = 0 + for row in rows: + reference = str(row["reference"][dimension]) + prediction = str(row["prediction"][dimension]) + confusion[(reference, prediction)] += 1 + matched = reference == prediction + correct += matched + stratum_counts[str(row["source_stratum"])][ + "correct" if matched else "incorrect" + ] += 1 + total = len(rows) + accuracy = correct / total if total else 0.0 + return { + "correct": correct, + "incorrect": total - correct, + "total": total, + "accuracy": round(accuracy, 6), + "target": target, + "passed": accuracy >= target, + "confusion": [ + { + "reference": reference, + "prediction": prediction, + "count": count, + } + for (reference, prediction), count in sorted( + confusion.items(), + key=lambda item: (-item[1], item[0]), + ) + ], + "by_stratum": { + stratum: { + "correct": counts["correct"], + "incorrect": counts["incorrect"], + "total": sum(counts.values()), + "accuracy": round(counts["correct"] / sum(counts.values()), 6), + } + for stratum, counts in sorted(stratum_counts.items()) + }, + } + + +def _validate_profile(profile: dict[str, Any]) -> None: + source = _object(profile.get("source"), "E39 source") + model = _object(profile.get("model"), "E39 model") + targets = _object(profile.get("targets"), "E39 targets") + if ( + profile.get("schema_version") != E39_PROFILE_SCHEMA + or profile.get("profile_id") + != "e39-ravnoves00-r1-development-knn/v1" + or source.get("session_id") != _SOURCE_SESSION_ID + or source.get("display_name") != _SOURCE_DISPLAY_NAME + or not isinstance(source.get("acceptance_result_id"), str) + or not str(source["acceptance_result_id"]).startswith( + "e37-ravnoves-acceptance-" + ) + or not isinstance(source.get("materialization_id"), str) + or not str(source["materialization_id"]).startswith("e30-materialization-") + or model.get("type") != "deterministic-robust-knn" + or model.get("feature_set") + != "source-scoped-camera-lidar-shape-v1" + or model.get("neighbors") != 3 + or model.get("robust_clip") != 20.0 + or model.get("cross_validation_folds") != 5 + or not isinstance(model.get("cross_validation_seed"), str) + or not model["cross_validation_seed"] + or any(targets.get(f"{name}_target") != 0.9 for name in _DIMENSIONS) + or targets.get("accounting_target") != 1.0 + or targets.get("maximum_false_free_claims") != 0 + or profile.get("authority") != _AUTHORITY + ): + raise E39PerceptionRefinementError("E39 profile contract changed") + + +def _span(value: object, axis: int) -> float: + if not isinstance(value, list) or len(value) != 2: + return -1.0 + if not all( + isinstance(item, list) + and len(item) > axis + and isinstance(item[axis], (int, float)) + for item in value + ): + return -1.0 + return float(value[1][axis]) - float(value[0][axis]) + + +def _number_or(value: object, fallback: float) -> float: + try: + parsed = float(value) # type: ignore[arg-type] + except (TypeError, ValueError): + return fallback + return parsed if math.isfinite(parsed) else fallback + + +def _rounded_vector(values: object) -> list[float]: + return [ + round(float(value), 12) + for value in np.asarray(values, dtype=np.float64).reshape(-1) + ] + + +def _artifact(path: Path, role: str) -> dict[str, Any]: + return { + "role": role, + "path": path.name, + "byte_length": path.stat().st_size, + "sha256": _sha256(path), + } + + +def _object(value: object, label: str) -> dict[str, Any]: + if not isinstance(value, dict): + raise E39PerceptionRefinementError(f"{label} must be an object") + return value + + +def _read_json(path: Path) -> dict[str, Any]: + try: + value = json.loads(path.read_text(encoding="utf-8-sig")) + except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc: + raise E39PerceptionRefinementError(f"invalid JSON: {path.name}") from exc + return _object(value, path.name) + + +def _read_jsonl(path: Path) -> list[dict[str, Any]]: + rows: list[dict[str, Any]] = [] + try: + with path.open("r", encoding="utf-8-sig") as stream: + for line in stream: + rows.append(_object(json.loads(line), path.name)) + except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc: + raise E39PerceptionRefinementError(f"invalid JSONL: {path.name}") from exc + return rows + + +def _write_json(path: Path, value: object) -> None: + with path.open("x", encoding="utf-8", newline="\n") as stream: + json.dump(value, stream, ensure_ascii=False, indent=2, sort_keys=True) + stream.write("\n") + stream.flush() + os.fsync(stream.fileno()) + + +def _write_jsonl(path: Path, rows: list[dict[str, Any]]) -> None: + with path.open("x", encoding="utf-8", newline="\n") as stream: + for row in rows: + stream.write( + json.dumps( + row, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + allow_nan=False, + ) + ) + stream.write("\n") + stream.flush() + os.fsync(stream.fileno()) + + +def _canonical_json(value: object) -> bytes: + return json.dumps( + value, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + allow_nan=False, + ).encode() + + +def _sha256(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as stream: + while chunk := stream.read(1024 * 1024): + digest.update(chunk) + return digest.hexdigest() + + +def _utc_now() -> str: + return datetime.now(UTC).isoformat(timespec="milliseconds").replace("+00:00", "Z") diff --git a/tests/test_e39_perception_refinement.py b/tests/test_e39_perception_refinement.py new file mode 100644 index 0000000..7830298 --- /dev/null +++ b/tests/test_e39_perception_refinement.py @@ -0,0 +1,57 @@ +from __future__ import annotations + +import numpy as np + +from k1link.compute.e39_perception_refinement import ( + _feature_names, + _fit_robust_scaler, + _predict_presence, + _project_dimensions, + _transform, +) + + +def test_e39_feature_contract_and_knn_are_deterministic() -> None: + assert len(_feature_names()) == 262 + matrix = np.asarray( + [ + [0.0, 1.0], + [0.1, 1.1], + [4.9, 5.0], + [5.0, 5.1], + ], + dtype=np.float64, + ) + median, scale = _fit_robust_scaler(matrix) + transformed = _transform(matrix, median=median, scale=scale, clip=20.0) + first = _predict_presence( + transformed[0], + transformed[1:], + ["background", "object", "object"], + neighbors=3, + ) + second = _predict_presence( + transformed[0], + transformed[1:], + ["background", "object", "object"], + neighbors=3, + ) + assert first == second == ("object", 2 / 3) + + +def test_e39_projects_presence_through_frozen_stratum_ontology() -> None: + assert _project_dimensions("camera-only", "object-present") == { + "presence": "object-present", + "geometry_association": "insufficient-support", + "freshness": "unavailable", + } + assert _project_dimensions("geometry-only", "occupied-environment") == { + "presence": "occupied-environment", + "geometry_association": "independent-occupied", + "freshness": "current", + } + assert _project_dimensions("unknown", "background-or-noise") == { + "presence": "background-or-noise", + "geometry_association": "rejected-nonobject", + "freshness": "current", + } diff --git a/tests/test_e39_worker_package.py b/tests/test_e39_worker_package.py new file mode 100644 index 0000000..fb4fe85 --- /dev/null +++ b/tests/test_e39_worker_package.py @@ -0,0 +1,73 @@ +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + + +def _module() -> object: + path = ( + Path(__file__).resolve().parents[1] + / "experiments" + / "perception" + / "prepare_e39_worker_package.py" + ) + spec = importlib.util.spec_from_file_location("e39_worker_package_test", path) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + sys.modules[spec.name] = module + spec.loader.exec_module(module) + return module + + +def test_e39_package_contains_bound_camera_lidar_evidence(tmp_path: Path) -> None: + module = _module() + repository = Path(__file__).resolve().parents[1] + acceptance = ( + repository + / ".runtime" + / "compute-experiments" + / "e37" + / "results" + / ( + "e37-ravnoves-acceptance-" + "01b1efd586f747341c712d82f0907b39436a6f91ae92b1dfae987eca05fd8344" + ) + ) + materialization = ( + repository + / ".runtime" + / "compute-experiments" + / "e30" + / "materializations" + / ( + "e30-materialization-" + "841af926d8d28ab93538c46d8f31278a2234c4d1c12c7dc4dc296b249d59735a" + ) + ) + package = module.build_e39_worker_package( + repository_root=repository, + acceptance_root=acceptance, + materialization_root=materialization, + profile_path=( + repository + / "experiments" + / "perception" + / "e39_ravnoves00_r1_refinement_profile.json" + ), + output_root=tmp_path, + ) + manifest = module.validate_e39_worker_package(package) + assert package.name == f"e39-worker-package-{manifest['identity_sha256']}" + assert manifest["identity"]["classification"] == ( + "immutable-ravnoves00-r1-camera-lidar-worker-input" + ) + assert len(manifest["artifacts"]) > 800 + assert ( + package + / "input" + / "materialization" + / materialization.name + / "frames" + / "frame-001005.jpg" + ).is_file()