feat(lab): measure RAVNOVES00 R1 quality baseline
This commit is contained in:
DCCONSTRUCTIONS
@@ -0,0 +1,129 @@
|
||||
# LAB E38 — RAVNOVES00 R1 perception-quality baseline
|
||||
|
||||
Date: 2026-07-28
|
||||
|
||||
Status: completed baseline; R1 quality gate not passed
|
||||
|
||||
## Why this LAB exists
|
||||
|
||||
E37 froze the RAVNOVES00 acceptance ontology, reviewed denominator and
|
||||
development/validation split. E38 is the first planned R1 measurement against
|
||||
that contract. It answers three separate questions:
|
||||
|
||||
1. Does the current source-scoped evidence support that a task-relevant object
|
||||
or occupied environment is present?
|
||||
2. Is current geometry associated with the correct object, rejected as
|
||||
non-object support, or retained as an independent occupied component?
|
||||
3. Is evidence freshness represented as current, stale or unavailable?
|
||||
|
||||
E38 does not evaluate navigation, planning, commands, safety or transfer to
|
||||
another route or rig.
|
||||
|
||||
## Immutable inputs
|
||||
|
||||
| Input | Identity |
|
||||
| --- | --- |
|
||||
| Physical source | `20260720T065719Z_viewer_live` (`RAVNOVES00`) |
|
||||
| Frozen E37 contract | `e37-ravnoves-acceptance-01b1efd586f747341c712d82f0907b39436a6f91ae92b1dfae987eca05fd8344` |
|
||||
| E37 item digest | `c6e4f474f59867ce0cc86825193043245c227106400cc216a2dac0645dca54ab` |
|
||||
| E30 materialization | `e30-materialization-841af926d8d28ab93538c46d8f31278a2234c4d1c12c7dc4dc296b249d59735a` |
|
||||
| E30 item digest | `827b498c7b7ab520cc3998e288134ebb5a2e8ebba0532d5cc3905e1a4460b0a7` |
|
||||
| E38 profile | `e38-ravnoves00-r1-development-cart/v1` |
|
||||
| Profile digest | `f34012fd6fedd2a8c3e5c6866cb2e57da9cb64bd6e798772da2c338b8a9b96c1` |
|
||||
|
||||
The E37 denominator contains 486 reviewed items: 340 development items and
|
||||
146 sealed validation items. Validation labels are not used for fitting or
|
||||
tree selection.
|
||||
|
||||
## Method
|
||||
|
||||
E38 joins each immutable E37 item with its immutable E30 materialization and
|
||||
derives deterministic source features. Three dimension-specific shallow CART
|
||||
models are trained only on the development partition:
|
||||
|
||||
- presence: maximum depth 5, minimum leaf 4;
|
||||
- geometry association: maximum depth 5, minimum leaf 4;
|
||||
- freshness: maximum depth 3, minimum leaf 6 and source-time features enabled.
|
||||
|
||||
Each validation item receives one terminal prediction for each dimension.
|
||||
Targets are evaluated separately: presence `>= 90%`, geometry association
|
||||
`>= 90%`, freshness `>= 90%`, accounting `100%`, false-free claims `0` and
|
||||
high-severity failures `0`.
|
||||
|
||||
This is an engineering-labelled source-scoped baseline, not an independent
|
||||
external ground-truth benchmark.
|
||||
|
||||
## Worker execution
|
||||
|
||||
| Property | Value |
|
||||
| --- | --- |
|
||||
| Worker | `DESKTOP-OPJ8J04` (`Worker 006`) |
|
||||
| Worker package | `e38-worker-package-a8b2313e07bd4d1de04db86de7728cbc21f89ae5259b2ee0adfb297eea6974c3` |
|
||||
| Container | `ndc-mission-core-e38-baseline` |
|
||||
| Pinned image | `nvcr.io/nvidia/tritonserver:26.06-py3@sha256:58df7489c3f2276f9591d500a012dee03e23d35543ce3c390b4c001e6bf90794` |
|
||||
| Network | disabled |
|
||||
| Limits | 2 CPU, 512 MiB memory, 128 PIDs |
|
||||
| Filesystem | read-only runtime, explicit output mount, 64 MiB no-exec tmpfs |
|
||||
| Authority | commands false; navigation/safety acceptance false |
|
||||
|
||||
The result was copied back from the Worker 006 D-volume and independently
|
||||
validated against its manifest and artifact digests.
|
||||
|
||||
## Result
|
||||
|
||||
Official result:
|
||||
|
||||
`e38-perception-baseline-a272f82988cd9a7e071fad94c3e9fb49daf804fdcca523f853445fd3113a62b1`
|
||||
|
||||
| Dimension | Correct | Accuracy | Target | Gate |
|
||||
| --- | ---: | ---: | ---: | --- |
|
||||
| Presence | 120 / 146 | 82.2% | >= 90% | fail |
|
||||
| Geometry association | 119 / 146 | 81.5% | >= 90% | fail |
|
||||
| Freshness | 139 / 146 | 95.2% | >= 90% | pass |
|
||||
| Evidence accounting | 146 / 146 | 100% | 100% | pass |
|
||||
| False-free claims | 0 | — | 0 | pass |
|
||||
| High-severity failures | 14 | — | 0 | fail |
|
||||
|
||||
The R1 quality gate is not passed.
|
||||
|
||||
### Weakest strata
|
||||
|
||||
- Presence is weakest on `geometry-only`: 25 / 38, or 65.8%.
|
||||
- Geometry association is weakest on `geometry-only`: 26 / 38, or 68.4%.
|
||||
- Freshness is weakest on `camera-only`: 34 / 39, or 87.2%.
|
||||
|
||||
Presence errors are dominated by background/noise being promoted to
|
||||
object-present and real objects being reduced to occupied environment.
|
||||
Geometry errors are dominated by object-associated evidence being retained as
|
||||
independent occupied geometry.
|
||||
|
||||
## What E38 proves
|
||||
|
||||
- RAVNOVES00 now has a reproducible first R1 baseline over a frozen holdout.
|
||||
- Freshness already exceeds the source-scoped 90% target.
|
||||
- All 146 validation cases receive terminal outcomes.
|
||||
- No missing or rejected evidence is converted into asserted free space.
|
||||
- The exact worker, immutable package, input identities, model and predictions
|
||||
are retained.
|
||||
|
||||
## What E38 does not prove
|
||||
|
||||
- Presence and geometry association have not reached 90%.
|
||||
- Fourteen high-severity validation items remain blocking.
|
||||
- E38 does not prove performance on another route, camera, mount or K1 unit.
|
||||
- E38 grants no navigation, command, planning or safety authority.
|
||||
|
||||
## Decision and next iteration
|
||||
|
||||
Keep E38 unchanged as the first measured R1 baseline. The next R1 LAB must
|
||||
focus on:
|
||||
|
||||
1. detector/background presence decisions;
|
||||
2. separating class-bearing objects from independent `geometry-only`
|
||||
occupied environment;
|
||||
3. the camera-only freshness tail;
|
||||
4. eliminating high-severity failures.
|
||||
|
||||
The 146 validation items remain frozen. Any model or rule change is developed
|
||||
only on the 340 development items and is published as a new LAB identity
|
||||
before the validation gate is evaluated again.
|
||||
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"authority": {
|
||||
"commands_enabled": false,
|
||||
"navigation_or_safety_accepted": false
|
||||
},
|
||||
"model": {
|
||||
"dimensions": {
|
||||
"freshness": {
|
||||
"include_source_time": true,
|
||||
"max_depth": 3,
|
||||
"min_leaf": 6
|
||||
},
|
||||
"geometry_association": {
|
||||
"include_source_time": false,
|
||||
"max_depth": 5,
|
||||
"min_leaf": 4
|
||||
},
|
||||
"presence": {
|
||||
"include_source_time": false,
|
||||
"max_depth": 5,
|
||||
"min_leaf": 4
|
||||
}
|
||||
},
|
||||
"type": "deterministic-shallow-cart"
|
||||
},
|
||||
"profile_id": "e38-ravnoves00-r1-development-cart/v1",
|
||||
"schema_version": "missioncore.e38-perception-baseline-profile/v1",
|
||||
"source": {
|
||||
"acceptance_result_id": "e37-ravnoves-acceptance-01b1efd586f747341c712d82f0907b39436a6f91ae92b1dfae987eca05fd8344",
|
||||
"display_name": "RAVNOVES00",
|
||||
"materialization_id": "e30-materialization-841af926d8d28ab93538c46d8f31278a2234c4d1c12c7dc4dc296b249d59735a",
|
||||
"session_id": "20260720T065719Z_viewer_live"
|
||||
},
|
||||
"targets": {
|
||||
"accounting_target": 1.0,
|
||||
"freshness_target": 0.9,
|
||||
"geometry_association_target": 0.9,
|
||||
"maximum_false_free_claims": 0,
|
||||
"presence_target": 0.9
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,292 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build a minimal immutable E38 package for Worker 006."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from k1link.compute.e38_perception_baseline import (
|
||||
E38_PACKAGE_SCHEMA,
|
||||
E38_PROFILE_SCHEMA,
|
||||
)
|
||||
|
||||
_RUNTIME_FILES = {
|
||||
"runtime/k1link/__init__.py": "src/k1link/__init__.py",
|
||||
"runtime/k1link/compute/__init__.py": None,
|
||||
"runtime/k1link/compute/e37_acceptance_contract.py": (
|
||||
"src/k1link/compute/e37_acceptance_contract.py"
|
||||
),
|
||||
"runtime/k1link/compute/e38_perception_baseline.py": (
|
||||
"src/k1link/compute/e38_perception_baseline.py"
|
||||
),
|
||||
"runtime/run_e38_perception_baseline.py": (
|
||||
"experiments/perception/worker/run_e38_perception_baseline.py"
|
||||
),
|
||||
"runtime/Invoke-E38PerceptionBaseline.ps1": (
|
||||
"experiments/perception/worker/Invoke-E38PerceptionBaseline.ps1"
|
||||
),
|
||||
}
|
||||
_GENERATED_COMPUTE_INIT = (
|
||||
'"""Minimal E38 worker projection; import contract modules explicitly."""\n'
|
||||
)
|
||||
_INPUT_FILES = {
|
||||
"acceptance": (
|
||||
"manifest.json",
|
||||
"acceptance-items.jsonl",
|
||||
"acceptance-contract.json",
|
||||
"run-report.json",
|
||||
),
|
||||
"materialization": ("manifest.json", "materialized-items.jsonl"),
|
||||
}
|
||||
|
||||
|
||||
class E38WorkerPackageError(RuntimeError):
|
||||
"""The E38 package source or immutable package is invalid."""
|
||||
|
||||
|
||||
def build_e38_worker_package(
|
||||
*,
|
||||
repository_root: Path,
|
||||
acceptance_root: Path,
|
||||
materialization_root: Path,
|
||||
profile_path: Path,
|
||||
output_root: Path,
|
||||
) -> Path:
|
||||
"""Build or verify one content-addressed E38 worker package."""
|
||||
|
||||
repository = repository_root.resolve(strict=True)
|
||||
profile_source = profile_path.resolve(strict=True)
|
||||
profile = _read_json(profile_source)
|
||||
if profile.get("schema_version") != E38_PROFILE_SCHEMA:
|
||||
raise E38WorkerPackageError("E38 package profile is incompatible")
|
||||
roots = {
|
||||
"acceptance": acceptance_root.resolve(strict=True),
|
||||
"materialization": materialization_root.resolve(strict=True),
|
||||
}
|
||||
expected_ids = {
|
||||
"acceptance": profile["source"]["acceptance_result_id"],
|
||||
"materialization": profile["source"]["materialization_id"],
|
||||
}
|
||||
sources: dict[str, Path | None] = {}
|
||||
for target, relative in _RUNTIME_FILES.items():
|
||||
source = None if relative is None else repository / relative
|
||||
if source is not None and (not source.is_file() or source.is_symlink()):
|
||||
raise E38WorkerPackageError(f"E38 runtime source is invalid: {relative}")
|
||||
sources[target] = source
|
||||
sources["profile.json"] = profile_source
|
||||
for kind, filenames in _INPUT_FILES.items():
|
||||
root = roots[kind]
|
||||
if root.name != expected_ids[kind]:
|
||||
raise E38WorkerPackageError(f"E38 {kind} identity changed")
|
||||
for filename in filenames:
|
||||
source = root / filename
|
||||
if not source.is_file() or source.is_symlink():
|
||||
raise E38WorkerPackageError(f"E38 {kind} artifact is invalid")
|
||||
sources[f"input/{kind}/{root.name}/{filename}"] = source
|
||||
|
||||
descriptors = []
|
||||
for relative, source in sorted(sources.items()):
|
||||
payload = (
|
||||
_GENERATED_COMPUTE_INIT.encode()
|
||||
if source is None
|
||||
else source.read_bytes()
|
||||
)
|
||||
descriptors.append(
|
||||
{
|
||||
"path": relative,
|
||||
"byte_length": len(payload),
|
||||
"sha256": hashlib.sha256(payload).hexdigest(),
|
||||
}
|
||||
)
|
||||
identity = {
|
||||
"schema_version": E38_PACKAGE_SCHEMA,
|
||||
"classification": "immutable-ravnoves00-r1-worker-input",
|
||||
"source_ids": expected_ids,
|
||||
"profile_sha256": _sha256(profile_source),
|
||||
"artifact_paths": [row["path"] for row in descriptors],
|
||||
"source_artifacts": descriptors,
|
||||
"authority": {
|
||||
"commands_enabled": False,
|
||||
"navigation_or_safety_accepted": False,
|
||||
},
|
||||
}
|
||||
identity_sha256 = hashlib.sha256(_canonical_json(identity)).hexdigest()
|
||||
package_id = f"e38-worker-package-{identity_sha256}"
|
||||
output = output_root.expanduser().absolute()
|
||||
output.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
destination = output / package_id
|
||||
if destination.exists():
|
||||
validate_e38_worker_package(destination)
|
||||
return destination
|
||||
|
||||
staging = output / f".{package_id}.{uuid.uuid4().hex}.tmp"
|
||||
staging.mkdir(mode=0o700, exist_ok=False)
|
||||
try:
|
||||
for relative, source in sources.items():
|
||||
target = staging / relative
|
||||
target.parent.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
if source is None:
|
||||
target.write_text(_GENERATED_COMPUTE_INIT, encoding="utf-8")
|
||||
else:
|
||||
shutil.copyfile(source, target)
|
||||
artifacts = [
|
||||
{
|
||||
"kind": relative,
|
||||
"path": relative,
|
||||
"byte_length": (staging / relative).stat().st_size,
|
||||
"sha256": _sha256(staging / relative),
|
||||
}
|
||||
for relative in sorted(sources)
|
||||
]
|
||||
manifest = {
|
||||
"schema_version": E38_PACKAGE_SCHEMA,
|
||||
"package_id": package_id,
|
||||
"identity_sha256": identity_sha256,
|
||||
"identity": identity,
|
||||
"created_at_utc": datetime.now(UTC)
|
||||
.isoformat(timespec="milliseconds")
|
||||
.replace("+00:00", "Z"),
|
||||
"artifacts": artifacts,
|
||||
}
|
||||
_write_json(staging / "manifest.json", manifest)
|
||||
validate_e38_worker_package(staging, allow_staging=True)
|
||||
os.replace(staging, destination)
|
||||
except BaseException:
|
||||
shutil.rmtree(staging, ignore_errors=True)
|
||||
raise
|
||||
validate_e38_worker_package(destination)
|
||||
return destination
|
||||
|
||||
|
||||
def validate_e38_worker_package(
|
||||
root: Path,
|
||||
*,
|
||||
allow_staging: bool = False,
|
||||
) -> dict[str, Any]:
|
||||
"""Validate package identity, exact file set, and every member digest."""
|
||||
|
||||
resolved = root.resolve(strict=True)
|
||||
manifest = _read_json(resolved / "manifest.json")
|
||||
identity = manifest.get("identity")
|
||||
identity_sha256 = manifest.get("identity_sha256")
|
||||
package_id = manifest.get("package_id")
|
||||
artifacts = manifest.get("artifacts")
|
||||
expected_name = (
|
||||
isinstance(package_id, str)
|
||||
and (
|
||||
resolved.name == package_id
|
||||
or (
|
||||
allow_staging
|
||||
and resolved.name.startswith(f".{package_id}.")
|
||||
and resolved.name.endswith(".tmp")
|
||||
)
|
||||
)
|
||||
)
|
||||
if (
|
||||
manifest.get("schema_version") != E38_PACKAGE_SCHEMA
|
||||
or not isinstance(identity, dict)
|
||||
or not isinstance(identity_sha256, str)
|
||||
or hashlib.sha256(_canonical_json(identity)).hexdigest() != identity_sha256
|
||||
or package_id != f"e38-worker-package-{identity_sha256}"
|
||||
or not expected_name
|
||||
or not isinstance(artifacts, list)
|
||||
):
|
||||
raise E38WorkerPackageError("E38 worker package identity is invalid")
|
||||
expected_paths = set(identity.get("artifact_paths", []))
|
||||
actual_paths = {
|
||||
path.relative_to(resolved).as_posix()
|
||||
for path in resolved.rglob("*")
|
||||
if path.is_file()
|
||||
}
|
||||
if (
|
||||
not expected_paths
|
||||
or actual_paths != expected_paths | {"manifest.json"}
|
||||
or len(artifacts) != len(expected_paths)
|
||||
):
|
||||
raise E38WorkerPackageError("E38 worker package file set changed")
|
||||
observed: set[str] = set()
|
||||
for row in artifacts:
|
||||
if not isinstance(row, dict):
|
||||
raise E38WorkerPackageError("E38 worker package artifact is invalid")
|
||||
relative = row.get("path")
|
||||
path = resolved / str(relative)
|
||||
if (
|
||||
not isinstance(relative, str)
|
||||
or relative not in expected_paths
|
||||
or relative in observed
|
||||
or Path(relative).is_absolute()
|
||||
or ".." in Path(relative).parts
|
||||
or not path.is_file()
|
||||
or path.is_symlink()
|
||||
or row.get("kind") != relative
|
||||
or row.get("byte_length") != path.stat().st_size
|
||||
or row.get("sha256") != _sha256(path)
|
||||
):
|
||||
raise E38WorkerPackageError("E38 worker package artifact changed")
|
||||
observed.add(relative)
|
||||
if observed != expected_paths:
|
||||
raise E38WorkerPackageError("E38 worker package coverage changed")
|
||||
return manifest
|
||||
|
||||
|
||||
def _canonical_json(value: object) -> bytes:
|
||||
return json.dumps(
|
||||
value,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
allow_nan=False,
|
||||
).encode()
|
||||
|
||||
|
||||
def _sha256(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as stream:
|
||||
while chunk := stream.read(1024 * 1024):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _read_json(path: Path) -> dict[str, Any]:
|
||||
value = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
if not isinstance(value, dict):
|
||||
raise E38WorkerPackageError(f"JSON object expected: {path.name}")
|
||||
return value
|
||||
|
||||
|
||||
def _write_json(path: Path, value: object) -> None:
|
||||
with path.open("x", encoding="utf-8", newline="\n") as stream:
|
||||
json.dump(value, stream, indent=2, sort_keys=True)
|
||||
stream.write("\n")
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--repository-root", type=Path, required=True)
|
||||
parser.add_argument("--acceptance", type=Path, required=True)
|
||||
parser.add_argument("--materialization", type=Path, required=True)
|
||||
parser.add_argument("--profile", type=Path, required=True)
|
||||
parser.add_argument("--output-root", type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
package = build_e38_worker_package(
|
||||
repository_root=args.repository_root,
|
||||
acceptance_root=args.acceptance,
|
||||
materialization_root=args.materialization,
|
||||
profile_path=args.profile,
|
||||
output_root=args.output_root,
|
||||
)
|
||||
print(package)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,50 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build the first RAVNOVES00 R1 perception-quality baseline."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from k1link.compute.e38_perception_baseline import (
|
||||
build_e38_perception_baseline,
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--acceptance", type=Path, required=True)
|
||||
parser.add_argument("--materialization", type=Path, required=True)
|
||||
parser.add_argument("--profile", type=Path, required=True)
|
||||
parser.add_argument("--output-root", type=Path, required=True)
|
||||
parser.add_argument("--worker-node", default=os.environ.get("COMPUTERNAME"))
|
||||
args = parser.parse_args()
|
||||
result = build_e38_perception_baseline(
|
||||
acceptance_root=args.acceptance,
|
||||
materialization_root=args.materialization,
|
||||
profile_path=args.profile,
|
||||
output_root=args.output_root,
|
||||
worker_node=args.worker_node,
|
||||
)
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"result_id": result.result_id,
|
||||
"result_root": str(result.result_root),
|
||||
"quality_gate_passed": result.quality_gate_passed,
|
||||
"metrics": result.report["metrics"],
|
||||
"blocking_checks": result.report["quality_gate"][
|
||||
"blocking_checks"
|
||||
],
|
||||
},
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
)
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,138 @@
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string]$PackageRoot,
|
||||
[string]$OutputRoot = "D:\NDC_MISSIONCORE\runtime\derived\e38-baseline",
|
||||
[string]$ContainerImage = "nvcr.io/nvidia/tritonserver:26.06-py3@sha256:58df7489c3f2276f9591d500a012dee03e23d35543ce3c390b4c001e6bf90794",
|
||||
[ValidateRange(1, 1000)]
|
||||
[int]$FreeGiBFloor = 300
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
$ProgressPreference = "SilentlyContinue"
|
||||
|
||||
function Assert-LastExitCode([string]$Operation) {
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "$Operation failed with exit code $LASTEXITCODE"
|
||||
}
|
||||
}
|
||||
|
||||
function Resolve-DDirectory([string]$Path, [string]$Label) {
|
||||
$item = Get-Item -LiteralPath (Resolve-Path -LiteralPath $Path).Path -Force
|
||||
$root = [IO.Path]::GetPathRoot($item.FullName).TrimEnd("\")
|
||||
if (
|
||||
-not $item.PSIsContainer -or
|
||||
($item.Attributes -band [IO.FileAttributes]::ReparsePoint) -or
|
||||
$root -ine "D:"
|
||||
) {
|
||||
throw "$Label must be a real D: directory"
|
||||
}
|
||||
return $item.FullName
|
||||
}
|
||||
|
||||
function Convert-ToDockerPath([string]$Path) {
|
||||
return $Path.Replace("\", "/")
|
||||
}
|
||||
|
||||
function Assert-FreeSpace([string]$Phase) {
|
||||
$free = [int64](Get-PSDrive -Name D).Free
|
||||
$floor = [int64]$FreeGiBFloor * 1GB
|
||||
Write-Host (
|
||||
"DISK_GUARD PHASE={0} DRIVE=D FREE_BYTES={1} FREE_GIB={2} FLOOR_GIB={3}" -f
|
||||
$Phase, $free, [math]::Round($free / 1GB, 3), $FreeGiBFloor
|
||||
)
|
||||
if ($free -lt ($floor + 1GB)) {
|
||||
throw "D: lacks the guarded E38 reserve during $Phase"
|
||||
}
|
||||
return $free
|
||||
}
|
||||
|
||||
$package = Resolve-DDirectory $PackageRoot "E38 package"
|
||||
$packageManifestPath = Join-Path $package "manifest.json"
|
||||
if (-not (Test-Path -LiteralPath $packageManifestPath -PathType Leaf)) {
|
||||
throw "E38 package manifest is missing"
|
||||
}
|
||||
$packageManifest = Get-Content -LiteralPath $packageManifestPath -Raw |
|
||||
ConvertFrom-Json
|
||||
if (
|
||||
$packageManifest.schema_version -ne "missioncore.e38-worker-package/v1" -or
|
||||
$packageManifest.package_id -ne (Split-Path $package -Leaf) -or
|
||||
$packageManifest.package_id -notmatch "^e38-worker-package-[a-f0-9]{64}$"
|
||||
) {
|
||||
throw "E38 package manifest is incompatible"
|
||||
}
|
||||
|
||||
if (-not (Test-Path -LiteralPath $OutputRoot)) {
|
||||
$null = New-Item -ItemType Directory -Path $OutputRoot
|
||||
}
|
||||
$output = Resolve-DDirectory $OutputRoot "E38 output root"
|
||||
$freeBefore = Assert-FreeSpace "preflight"
|
||||
|
||||
& docker image inspect $ContainerImage *> $null
|
||||
Assert-LastExitCode "Pinned E38 container image inspection"
|
||||
|
||||
$dockerPackage = Convert-ToDockerPath $package
|
||||
$dockerOutput = Convert-ToDockerPath $output
|
||||
$packageName = Split-Path $package -Leaf
|
||||
$containerPackage = "/opt/e38-input/$packageName"
|
||||
$command = @(
|
||||
"run", "--rm",
|
||||
"--name", "ndc-mission-core-e38-baseline",
|
||||
"--network", "none",
|
||||
"--read-only",
|
||||
"--security-opt", "no-new-privileges:true",
|
||||
"--cap-drop", "ALL",
|
||||
"--pids-limit", "128",
|
||||
"--memory", "512m",
|
||||
"--memory-swap", "512m",
|
||||
"--cpus", "2",
|
||||
"--tmpfs", "/tmp:rw,noexec,nosuid,size=64m",
|
||||
"-e", "PYTHONDONTWRITEBYTECODE=1",
|
||||
"-e", ("PYTHONPATH={0}/runtime" -f $containerPackage),
|
||||
"-e", ("E38_WORKER_NODE={0}" -f $env:COMPUTERNAME),
|
||||
"-v", ("{0}:{1}:ro" -f $dockerPackage, $containerPackage),
|
||||
"-v", ("{0}:/output:rw" -f $dockerOutput),
|
||||
"--entrypoint", "python3",
|
||||
$ContainerImage,
|
||||
("{0}/runtime/run_e38_perception_baseline.py" -f $containerPackage),
|
||||
"--package", $containerPackage,
|
||||
"--output-root", "/output"
|
||||
)
|
||||
|
||||
Write-Output ("PACKAGE_ID={0}" -f $packageManifest.package_id)
|
||||
Write-Output ("PACKAGE_IDENTITY_SHA256={0}" -f $packageManifest.identity_sha256)
|
||||
Write-Output ("CONTAINER_IMAGE={0}" -f $ContainerImage)
|
||||
& docker @command
|
||||
Assert-LastExitCode "E38 perception baseline"
|
||||
|
||||
$matches = @(
|
||||
Get-ChildItem -LiteralPath $output -Directory -Filter "e38-perception-baseline-*" |
|
||||
Where-Object {
|
||||
$manifestPath = Join-Path $_.FullName "manifest.json"
|
||||
if (-not (Test-Path -LiteralPath $manifestPath -PathType Leaf)) {
|
||||
return $false
|
||||
}
|
||||
$manifest = Get-Content -LiteralPath $manifestPath -Raw |
|
||||
ConvertFrom-Json
|
||||
return (
|
||||
$manifest.schema_version -eq
|
||||
"missioncore.e38-perception-baseline/v1" -and
|
||||
$manifest.acceptance_state -eq
|
||||
"completed-r1-source-scoped-baseline" -and
|
||||
$manifest.identity.execution.worker_node -eq $env:COMPUTERNAME
|
||||
)
|
||||
}
|
||||
)
|
||||
if ($matches.Count -ne 1) {
|
||||
throw "E38 immutable result could not be resolved uniquely"
|
||||
}
|
||||
$resultRoot = $matches[0].FullName
|
||||
$resultManifest = Get-Content -LiteralPath (
|
||||
Join-Path $resultRoot "manifest.json"
|
||||
) -Raw | ConvertFrom-Json
|
||||
$freeAfter = Assert-FreeSpace "completed"
|
||||
Write-Output ("RESULT_ROOT={0}" -f $resultRoot)
|
||||
Write-Output ("RESULT_ID={0}" -f $resultManifest.result_id)
|
||||
Write-Output ("QUALITY_GATE_PASSED={0}" -f $resultManifest.quality_gate_passed)
|
||||
Write-Output ("DISK_FREE_BYTES_BEFORE={0}" -f $freeBefore)
|
||||
Write-Output ("DISK_FREE_BYTES_AFTER={0}" -f $freeAfter)
|
||||
@@ -0,0 +1,48 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Execute a packaged E38 baseline inside the pinned Worker 006 container."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from k1link.compute.e38_perception_baseline import (
|
||||
build_e38_perception_baseline,
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--package", type=Path, required=True)
|
||||
parser.add_argument("--output-root", type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
package = args.package.resolve(strict=True)
|
||||
manifest = json.loads((package / "manifest.json").read_text(encoding="utf-8"))
|
||||
source = manifest["identity"]["source_ids"]
|
||||
result = build_e38_perception_baseline(
|
||||
acceptance_root=package / "input" / "acceptance" / source["acceptance"],
|
||||
materialization_root=(
|
||||
package / "input" / "materialization" / source["materialization"]
|
||||
),
|
||||
profile_path=package / "profile.json",
|
||||
output_root=args.output_root,
|
||||
worker_node=os.environ.get("E38_WORKER_NODE"),
|
||||
)
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"result_id": result.result_id,
|
||||
"quality_gate_passed": result.quality_gate_passed,
|
||||
"metrics": result.report["metrics"],
|
||||
},
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
)
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user