feat(simulation): qualify S0 time and resource gates
This commit is contained in:
DCCONSTRUCTIONS
parent
6290fcb2ce
commit
c9b1009259
|
|
@ -57,8 +57,10 @@ Polygon is now a parallel product branch. As of this document:
|
|||
PX4/Gazebo/ROS 2/Nav2 stack has been installed without using Docker;
|
||||
- PX4 v1.17.0 and the stock Ackermann rover have completed repeated factual
|
||||
smoke runs with ROS 2 vehicle status, Gazebo clock and clean shutdown;
|
||||
- the complete digest-bound S0 evidence manifest has not been admitted;
|
||||
- pause/step/speed and accepted 1×/2× resource evidence remain open;
|
||||
- the exact accepted-profile generation has passed pause freeze, one 2 ms
|
||||
single-step, 1×/2× speed and structured resource cases;
|
||||
- the complete digest-bound S0 evidence manifest and final target doctor
|
||||
verdict are the remaining S0 admission steps;
|
||||
- `actuator_authority=false`;
|
||||
- `navigation_or_safety_accepted=false`.
|
||||
|
||||
|
|
@ -369,7 +371,7 @@ incompatible.
|
|||
|
||||
## 14. Upstream baseline
|
||||
|
||||
The S0 candidate line is:
|
||||
The S0 accepted line is:
|
||||
|
||||
- Windows 11 AI worker;
|
||||
- dedicated `MissionCore-Sim` WSL2 Ubuntu 24.04 distribution physically on D;
|
||||
|
|
@ -382,10 +384,11 @@ The S0 candidate line is:
|
|||
- Nav2 Jazzy;
|
||||
- PX4 Gazebo stock rover models.
|
||||
|
||||
The exact source commits and package versions are resolved in the candidate
|
||||
profile. Target installation, PX4 build, DDS telemetry and repeated stock-rover
|
||||
smoke evidence now exist, but components do not become `accepted` until the
|
||||
remaining S0 behavior/resource cases and digest-bound evidence generation pass.
|
||||
The exact source commits and package versions are resolved in the profile.
|
||||
Target installation, PX4 build, DDS telemetry, Nav2/ROS package discovery and
|
||||
repeated 1×/2× stock-rover evidence have moved every component pin to
|
||||
`accepted`. A future pin change creates a new profile generation and invalidates
|
||||
evidence tied to the prior SHA-256.
|
||||
|
||||
S1 uses direct exact `px4_msgs` integration. The experimental
|
||||
`px4-ros2-interface-lib` is evaluated later and is not on the S1 critical path.
|
||||
|
|
@ -415,6 +418,14 @@ effective exposure remains loopback-only. ROS domain 0 is isolated by the same
|
|||
boundary. This distinction is part of the port evidence and must not be
|
||||
misreported as a host-wide `127.0.0.1` bind.
|
||||
|
||||
The stock rover world declares a 2 ms physics step. S0 pause acceptance permits
|
||||
zero simulation-time advance; single-step requires exactly +2,000,000 ns and
|
||||
+1 iteration. Speed factors 1× and 2× are measured over four seconds with a
|
||||
±20% RTF tolerance. Resource sampling runs every 500 ms with an owned-process
|
||||
budget of 2 GiB RSS and 800% CPU, at least 8 GiB available WSL memory and the
|
||||
existing 200 GiB D stop floor. GPU values are recorded as global co-tenant
|
||||
context and are not misattributed to headless S0.
|
||||
|
||||
## 16. Delivery plan
|
||||
|
||||
| Phase | Estimate | Exit condition |
|
||||
|
|
|
|||
|
|
@ -90,6 +90,12 @@ providers, observes ROS 2 vehicle status and Gazebo clock, requests PX4 shutdown
|
|||
checks for owned-process residue and writes D-only factual artifacts. It never
|
||||
arms the vehicle or sends an actuator command.
|
||||
|
||||
`simulation/s0/gazebo_time_control.py` owns the S0 pause/single-step/1×/2× RTF
|
||||
and resource acceptance probe. `simulation/s0/build_evidence_manifest.py`
|
||||
admits only exact-profile, D-confined, passing 1×/2× runs into the digest-bound
|
||||
evidence generation. These scripts are qualification tooling; the S1
|
||||
Simulation Orchestrator remains the future product lifecycle owner.
|
||||
|
||||
The doctor reports `INCOMPLETE` until it is running on the reviewed target with
|
||||
resolved/accepted pins and all required evidence. It cannot infer `GO` from the
|
||||
repository state.
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ Copy the reviewed script to the D-only worker source root, then invoke it as
|
|||
root inside `MissionCore-Sim` with a unique immutable run ID:
|
||||
|
||||
```bash
|
||||
simulation/s0/worker-smoke.sh <UTC-run-id>-stock-rover
|
||||
simulation/s0/worker-smoke.sh <UTC-run-id>-stock-rover <1|2>
|
||||
```
|
||||
|
||||
The script:
|
||||
|
|
@ -44,11 +44,15 @@ The script:
|
|||
- creates a new network namespace containing only loopback;
|
||||
- starts Micro XRCE-DDS Agent, PX4 SITL and the stock Gazebo Ackermann world;
|
||||
- verifies the DDS session, `/fmu/out/vehicle_status_v1` and Gazebo `/clock`;
|
||||
- records listeners and a non-acceptance resource point sample;
|
||||
- proves pause freeze and exactly one 2 ms physics step;
|
||||
- measures the declared 1× or 2× real-time factor over four seconds;
|
||||
- samples owned CPU/RSS, available RAM, swap, D free space and global co-tenant
|
||||
GPU load at 500 ms intervals;
|
||||
- records Gazebo service/topic inventory and runtime listeners;
|
||||
- sends the PX4 `shutdown` command and rejects owned-process residue.
|
||||
|
||||
Do not reuse a run ID. A failed run remains diagnostic evidence; rerun with a
|
||||
new ID after correction.
|
||||
Run both factors as separate immutable cases. Do not reuse a run ID. A failed
|
||||
run remains diagnostic evidence; rerun with a new ID after correction.
|
||||
|
||||
## Required target inventory
|
||||
|
||||
|
|
@ -119,6 +123,25 @@ The manifest must contain exactly all IDs from `required_evidence`. Artifact
|
|||
paths are relative, confined regular files. The doctor rejects missing,
|
||||
additional, changed or cross-generation evidence.
|
||||
|
||||
Build the redacted manifest only from two passing runs made with the exact
|
||||
accepted profile generation:
|
||||
|
||||
```bash
|
||||
python3 simulation/s0/build_evidence_manifest.py \
|
||||
--profile /mnt/d/NDC_MISSIONCORE/simulation/source/qualification-profile.yaml \
|
||||
--run-1x /mnt/d/NDC_MISSIONCORE/simulation/artifacts/s0/<accepted-1x-run> \
|
||||
--run-2x /mnt/d/NDC_MISSIONCORE/simulation/artifacts/s0/<accepted-2x-run> \
|
||||
--output-dir /mnt/d/NDC_MISSIONCORE/simulation/artifacts/s0/evidence/<generation> \
|
||||
--d-root /mnt/d/NDC_MISSIONCORE \
|
||||
--source-root /mnt/d/NDC_MISSIONCORE/simulation/source \
|
||||
--repository-commit <full-commit-sha> \
|
||||
--wsl-windows-path 'D:\NDC_MISSIONCORE\simulation\wsl\MissionCore-Sim'
|
||||
```
|
||||
|
||||
The generator refuses an existing output directory, non-passing runs,
|
||||
candidate pins, commit drift, incomplete Nav2/ROS discovery or any input/output
|
||||
outside the reviewed D root.
|
||||
|
||||
Validate without writing:
|
||||
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -0,0 +1,479 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Build a confined, digest-bound SIM S0 evidence pack from accepted worker runs."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
EVIDENCE_SCHEMA = "missioncore.simulation-s0-evidence/v1"
|
||||
|
||||
|
||||
class EvidenceError(RuntimeError):
|
||||
"""Evidence inputs are incomplete, inconsistent, or outside the reviewed boundary."""
|
||||
|
||||
|
||||
def _run(command: list[str]) -> str:
|
||||
result = subprocess.run(
|
||||
command,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=15,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise EvidenceError(
|
||||
f"command failed ({result.returncode}): {' '.join(command)}; "
|
||||
f"stderr={result.stderr.strip()!r}"
|
||||
)
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def _load_json(path: Path) -> dict[str, Any]:
|
||||
loaded = json.loads(path.read_text(encoding="utf-8"))
|
||||
if not isinstance(loaded, dict):
|
||||
raise EvidenceError(f"expected a JSON object: {path}")
|
||||
return loaded
|
||||
|
||||
|
||||
def _load_yaml(path: Path) -> dict[str, Any]:
|
||||
loaded = yaml.safe_load(path.read_text(encoding="utf-8"))
|
||||
if not isinstance(loaded, dict):
|
||||
raise EvidenceError(f"expected a YAML mapping: {path}")
|
||||
return loaded
|
||||
|
||||
|
||||
def _result_env(path: Path) -> dict[str, str]:
|
||||
result: dict[str, str] = {}
|
||||
for line in path.read_text(encoding="utf-8").splitlines():
|
||||
key, separator, value = line.partition("=")
|
||||
if not separator or not key:
|
||||
raise EvidenceError(f"invalid result line in {path}: {line!r}")
|
||||
result[key] = value
|
||||
required_passes = {
|
||||
"px4_stock_rover",
|
||||
"uxrce_dds",
|
||||
"ros2_vehicle_status",
|
||||
"gazebo_clock",
|
||||
"pause_step_speed",
|
||||
"resource_baseline",
|
||||
"clean_lifecycle",
|
||||
}
|
||||
failed = sorted(key for key in required_passes if result.get(key) != "pass")
|
||||
if failed:
|
||||
raise EvidenceError(f"run did not pass required checks: {', '.join(failed)}")
|
||||
if result.get("actuator_authority") != "false":
|
||||
raise EvidenceError("run widened actuator authority")
|
||||
return result
|
||||
|
||||
|
||||
def _sha256(path: Path) -> str:
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def _write_json(path: Path, payload: dict[str, Any]) -> None:
|
||||
path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def _package_version(package: str) -> str:
|
||||
return _run(["dpkg-query", "-W", "-f=${Version}", package])
|
||||
|
||||
|
||||
def _git_commit(path: Path) -> str:
|
||||
commit = _run(["git", "-C", str(path), "rev-parse", "HEAD"])
|
||||
if re.fullmatch(r"[a-f0-9]{40}", commit) is None:
|
||||
raise EvidenceError(f"invalid Git commit at {path}: {commit!r}")
|
||||
return commit
|
||||
|
||||
|
||||
def _memory_inventory() -> dict[str, int]:
|
||||
values: dict[str, int] = {}
|
||||
for line in Path("/proc/meminfo").read_text(encoding="utf-8").splitlines():
|
||||
key, raw = line.split(":", maxsplit=1)
|
||||
values[key] = int(raw.strip().split()[0]) * 1024
|
||||
return {
|
||||
"total_bytes": values["MemTotal"],
|
||||
"available_bytes": values["MemAvailable"],
|
||||
"swap_total_bytes": values["SwapTotal"],
|
||||
"swap_free_bytes": values["SwapFree"],
|
||||
}
|
||||
|
||||
|
||||
def _gpu_inventory() -> dict[str, str | int] | None:
|
||||
if shutil.which("nvidia-smi") is None:
|
||||
return None
|
||||
output = _run(
|
||||
[
|
||||
"nvidia-smi",
|
||||
"--query-gpu=name,driver_version,memory.total",
|
||||
"--format=csv,noheader,nounits",
|
||||
]
|
||||
)
|
||||
name, driver, memory_mib = (part.strip() for part in output.splitlines()[0].split(","))
|
||||
return {
|
||||
"name": name,
|
||||
"driver": driver,
|
||||
"memory_total_mib": int(memory_mib),
|
||||
}
|
||||
|
||||
|
||||
def _cpu_model() -> str:
|
||||
for line in Path("/proc/cpuinfo").read_text(encoding="utf-8").splitlines():
|
||||
if line.startswith("model name"):
|
||||
return line.split(":", maxsplit=1)[1].strip()
|
||||
raise EvidenceError("CPU model was not available")
|
||||
|
||||
|
||||
def _component_inventory(profile: dict[str, Any], source_root: Path) -> dict[str, Any]:
|
||||
components = profile.get("components")
|
||||
if not isinstance(components, list) or not components:
|
||||
raise EvidenceError("profile components are missing")
|
||||
if any(
|
||||
not isinstance(component, dict) or component.get("qualification_state") != "accepted"
|
||||
for component in components
|
||||
):
|
||||
raise EvidenceError("all profile components must be accepted before evidence admission")
|
||||
|
||||
packages = {
|
||||
"ros-jazzy-ros-base": _package_version("ros-jazzy-ros-base"),
|
||||
"ros-jazzy-ros-gz": _package_version("ros-jazzy-ros-gz"),
|
||||
"gz-harmonic": _package_version("gz-harmonic"),
|
||||
"gz-sim8-cli": _package_version("gz-sim8-cli"),
|
||||
"libsdformat14": _package_version("libsdformat14"),
|
||||
"ros-jazzy-navigation2": _package_version("ros-jazzy-navigation2"),
|
||||
"ros-jazzy-nav2-bringup": _package_version("ros-jazzy-nav2-bringup"),
|
||||
}
|
||||
commits = {
|
||||
"px4-autopilot": _git_commit(source_root / "PX4-Autopilot"),
|
||||
"px4-msgs": _git_commit(source_root / "px4_msgs"),
|
||||
"px4-gazebo-models": _git_commit(
|
||||
source_root / "PX4-Autopilot" / "Tools" / "simulation" / "gz"
|
||||
),
|
||||
"micro-xrce-dds-agent": _git_commit(source_root / "Micro-XRCE-DDS-Agent"),
|
||||
}
|
||||
declared = {
|
||||
str(component["id"]): {
|
||||
"resolved_version": component.get("resolved_version"),
|
||||
"resolved_commit": component.get("resolved_commit"),
|
||||
"license": component.get("license"),
|
||||
"qualification_state": component.get("qualification_state"),
|
||||
}
|
||||
for component in components
|
||||
}
|
||||
for identifier, commit in commits.items():
|
||||
if declared[identifier]["resolved_commit"] != commit:
|
||||
raise EvidenceError(f"worker commit drift for {identifier}")
|
||||
|
||||
nav2_prefix = _run(["ros2", "pkg", "prefix", "nav2_bringup"])
|
||||
px4_msgs_prefix = _run(["ros2", "pkg", "prefix", "px4_msgs"])
|
||||
nav2_executables = _run(["ros2", "pkg", "executables", "nav2_controller"]).splitlines()
|
||||
if not nav2_executables:
|
||||
raise EvidenceError("Nav2 controller executable is unavailable")
|
||||
return {
|
||||
"declared": declared,
|
||||
"installed_packages": packages,
|
||||
"observed_commits": commits,
|
||||
"runtime_discovery": {
|
||||
"nav2_bringup_prefix": nav2_prefix,
|
||||
"nav2_controller_executables": nav2_executables,
|
||||
"px4_msgs_prefix": px4_msgs_prefix,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _world_model_from_log(path: Path) -> dict[str, str]:
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
world_model = re.search(r"world:\s*([^,\s]+),\s*model:\s*([^\s]+)", text)
|
||||
commit = re.search(r"PX4 git-hash:\s*([a-f0-9]{40})", text)
|
||||
version = re.search(r"PX4 version:\s*(.+)", text)
|
||||
if world_model is None or commit is None or version is None:
|
||||
raise EvidenceError(f"PX4 identity is incomplete in {path}")
|
||||
return {
|
||||
"world": world_model.group(1),
|
||||
"model": world_model.group(2),
|
||||
"px4_commit": commit.group(1),
|
||||
"px4_version": version.group(1).strip(),
|
||||
}
|
||||
|
||||
|
||||
def _ensure_confined(path: Path, root: Path, *, label: str) -> Path:
|
||||
resolved = path.expanduser().resolve(strict=True)
|
||||
if not resolved.is_relative_to(root):
|
||||
raise EvidenceError(f"{label} escaped reviewed root {root}: {resolved}")
|
||||
return resolved
|
||||
|
||||
|
||||
def _arguments() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--profile", type=Path, required=True)
|
||||
parser.add_argument("--run-1x", type=Path, required=True)
|
||||
parser.add_argument("--run-2x", type=Path, required=True)
|
||||
parser.add_argument("--output-dir", type=Path, required=True)
|
||||
parser.add_argument("--d-root", type=Path, required=True)
|
||||
parser.add_argument("--source-root", type=Path, required=True)
|
||||
parser.add_argument("--repository-commit", required=True)
|
||||
parser.add_argument("--wsl-windows-path", required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
args = _arguments()
|
||||
d_root = args.d_root.expanduser().resolve(strict=True)
|
||||
profile_path = _ensure_confined(args.profile, d_root, label="profile")
|
||||
run_1x = _ensure_confined(args.run_1x, d_root, label="1x run")
|
||||
run_2x = _ensure_confined(args.run_2x, d_root, label="2x run")
|
||||
source_root = _ensure_confined(args.source_root, d_root, label="source root")
|
||||
output_dir = args.output_dir.expanduser().resolve(strict=False)
|
||||
if not output_dir.is_relative_to(d_root):
|
||||
raise EvidenceError("evidence output escaped the reviewed D-only root")
|
||||
if output_dir.exists():
|
||||
raise EvidenceError(f"evidence output already exists: {output_dir}")
|
||||
if re.fullmatch(r"[a-f0-9]{40}", args.repository_commit) is None:
|
||||
raise EvidenceError("repository commit must be a full Git SHA")
|
||||
|
||||
profile = _load_yaml(profile_path)
|
||||
profile_sha256 = _sha256(profile_path)
|
||||
result_1x = _result_env(run_1x / "result.env")
|
||||
result_2x = _result_env(run_2x / "result.env")
|
||||
if result_1x.get("speed_factor") != "1" or result_2x.get("speed_factor") != "2":
|
||||
raise EvidenceError("run speed factors are not the declared 1x/2x pair")
|
||||
time_1x = _load_json(run_1x / "time-control.json")
|
||||
time_2x = _load_json(run_2x / "time-control.json")
|
||||
resource_1x = _load_json(run_1x / "resource-baseline.json")
|
||||
resource_2x = _load_json(run_2x / "resource-baseline.json")
|
||||
if any(
|
||||
report.get("verdict") != "pass" for report in (time_1x, time_2x, resource_1x, resource_2x)
|
||||
):
|
||||
raise EvidenceError("time or resource report is not a pass")
|
||||
|
||||
output_dir.mkdir(parents=True)
|
||||
disk = shutil.disk_usage(d_root)
|
||||
target_host = profile["target_host"]
|
||||
worker_inventory = {
|
||||
"redacted": True,
|
||||
"repository_commit": args.repository_commit,
|
||||
"profile_sha256": profile_sha256,
|
||||
"target": {
|
||||
"system": platform.system(),
|
||||
"kernel_release": platform.release(),
|
||||
"architecture": platform.machine(),
|
||||
"wsl_distribution": os.environ.get("WSL_DISTRO_NAME"),
|
||||
"linux_pretty_name": _run(
|
||||
["bash", "-lc", ". /etc/os-release && printf '%s' \"$PRETTY_NAME\""]
|
||||
),
|
||||
},
|
||||
"cpu": {
|
||||
"model": _cpu_model(),
|
||||
"logical_processors": os.cpu_count(),
|
||||
},
|
||||
"memory": _memory_inventory(),
|
||||
"gpu": _gpu_inventory(),
|
||||
"disk": {
|
||||
"total_bytes": disk.total,
|
||||
"free_bytes": disk.free,
|
||||
},
|
||||
"components": _component_inventory(profile, source_root),
|
||||
}
|
||||
if worker_inventory["target"]["wsl_distribution"] != target_host["wsl_distribution"]:
|
||||
raise EvidenceError("worker distribution does not match the profile")
|
||||
|
||||
mutable_paths = profile["storage"]["mutable_paths"]
|
||||
placement: list[dict[str, Any]] = []
|
||||
for item in mutable_paths:
|
||||
path = Path(str(item["wsl_path"])).resolve(strict=True)
|
||||
if not path.is_relative_to(d_root):
|
||||
raise EvidenceError(f"mutable path escaped D-only root: {path}")
|
||||
placement.append(
|
||||
{
|
||||
"id": item["id"],
|
||||
"windows_path": item["windows_path"],
|
||||
"wsl_path": str(path),
|
||||
"exists": path.is_dir(),
|
||||
}
|
||||
)
|
||||
d_only = {
|
||||
"redacted": True,
|
||||
"profile_sha256": profile_sha256,
|
||||
"wsl_distribution_windows_path": args.wsl_windows_path,
|
||||
"wsl_mount_root": str(d_root),
|
||||
"container_runtime": target_host["container_runtime"],
|
||||
"existing_docker_desktop_used_or_modified": False,
|
||||
"mutable_paths": placement,
|
||||
"disk": {
|
||||
"total_bytes": disk.total,
|
||||
"free_bytes": disk.free,
|
||||
"minimum_free_gib": profile["storage"]["minimum_free_gib"],
|
||||
"stop_below_gib": profile["storage"]["stop_below_gib"],
|
||||
},
|
||||
}
|
||||
|
||||
px4_launch = {
|
||||
"repository_commit": args.repository_commit,
|
||||
"runs": [
|
||||
{
|
||||
"run_id": result_1x["run_id"],
|
||||
"speed_factor": 1,
|
||||
"identity": _world_model_from_log(run_1x / "px4-rover.log"),
|
||||
"result": result_1x,
|
||||
},
|
||||
{
|
||||
"run_id": result_2x["run_id"],
|
||||
"speed_factor": 2,
|
||||
"identity": _world_model_from_log(run_2x / "px4-rover.log"),
|
||||
"result": result_2x,
|
||||
},
|
||||
],
|
||||
}
|
||||
telemetry = {
|
||||
"runs": [
|
||||
{
|
||||
"run_id": result["run_id"],
|
||||
"vehicle_status": _load_yaml(run / "vehicle-status.yaml"),
|
||||
"vehicle_status_topic_present": (
|
||||
"/fmu/out/vehicle_status_v1"
|
||||
in (run / "ros-topics.txt").read_text(encoding="utf-8")
|
||||
),
|
||||
}
|
||||
for result, run in ((result_1x, run_1x), (result_2x, run_2x))
|
||||
]
|
||||
}
|
||||
authoritative_clock = {
|
||||
"authority": profile["clock"],
|
||||
"runs": [
|
||||
{
|
||||
"run_id": result["run_id"],
|
||||
"clock_sample": _load_yaml(run / "clock.yaml"),
|
||||
"pause_advance_ns": report["pause"]["observed_advance_ns"],
|
||||
}
|
||||
for result, run, report in (
|
||||
(result_1x, run_1x, time_1x),
|
||||
(result_2x, run_2x, time_2x),
|
||||
)
|
||||
],
|
||||
}
|
||||
pause_step_speed = {
|
||||
"contract": profile["runtime_acceptance"],
|
||||
"runs": [
|
||||
{
|
||||
"run_id": result_1x["run_id"],
|
||||
"speed_factor": 1,
|
||||
"pause": time_1x["pause"],
|
||||
"single_step": time_1x["single_step"],
|
||||
"speed": time_1x["speed"],
|
||||
"verdict": time_1x["verdict"],
|
||||
},
|
||||
{
|
||||
"run_id": result_2x["run_id"],
|
||||
"speed_factor": 2,
|
||||
"pause": time_2x["pause"],
|
||||
"single_step": time_2x["single_step"],
|
||||
"speed": time_2x["speed"],
|
||||
"verdict": time_2x["verdict"],
|
||||
},
|
||||
],
|
||||
}
|
||||
clean_stop = {
|
||||
"runs": [
|
||||
{
|
||||
"run_id": result_1x["run_id"],
|
||||
"clean_lifecycle": result_1x["clean_lifecycle"],
|
||||
},
|
||||
{
|
||||
"run_id": result_2x["run_id"],
|
||||
"clean_lifecycle": result_2x["clean_lifecycle"],
|
||||
},
|
||||
],
|
||||
"owned_process_residue": False,
|
||||
}
|
||||
|
||||
artifacts: dict[str, tuple[str, dict[str, Any]]] = {
|
||||
"worker-inventory": (
|
||||
"worker-inventory.redacted.json",
|
||||
worker_inventory,
|
||||
),
|
||||
"d-only-physical-placement": (
|
||||
"d-only-physical-placement.redacted.json",
|
||||
d_only,
|
||||
),
|
||||
"px4-stock-rover-launch": (
|
||||
"px4-stock-rover-launch.redacted.json",
|
||||
px4_launch,
|
||||
),
|
||||
"ros2-telemetry": (
|
||||
"ros2-telemetry.redacted.json",
|
||||
telemetry,
|
||||
),
|
||||
"authoritative-simulation-clock": (
|
||||
"authoritative-simulation-clock.redacted.json",
|
||||
authoritative_clock,
|
||||
),
|
||||
"pause-step-speed": (
|
||||
"pause-step-speed.redacted.json",
|
||||
pause_step_speed,
|
||||
),
|
||||
"clean-stop-no-orphans": (
|
||||
"clean-stop-no-orphans.redacted.json",
|
||||
clean_stop,
|
||||
),
|
||||
"resource-baseline-1x": (
|
||||
"resource-baseline-1x.redacted.json",
|
||||
resource_1x,
|
||||
),
|
||||
"resource-baseline-2x": (
|
||||
"resource-baseline-2x.redacted.json",
|
||||
resource_2x,
|
||||
),
|
||||
}
|
||||
required = profile.get("required_evidence")
|
||||
if not isinstance(required, list) or set(required) != set(artifacts):
|
||||
raise EvidenceError("profile required_evidence does not match the generator")
|
||||
|
||||
manifest_checks: list[dict[str, str]] = []
|
||||
for identifier in required:
|
||||
filename, payload = artifacts[str(identifier)]
|
||||
artifact_path = output_dir / filename
|
||||
_write_json(artifact_path, payload)
|
||||
manifest_checks.append(
|
||||
{
|
||||
"id": str(identifier),
|
||||
"status": "pass",
|
||||
"artifact": filename,
|
||||
"sha256": _sha256(artifact_path),
|
||||
"note": f"Redacted factual worker evidence for {identifier}.",
|
||||
}
|
||||
)
|
||||
manifest = {
|
||||
"schema_version": EVIDENCE_SCHEMA,
|
||||
"profile_sha256": profile_sha256,
|
||||
"checks": manifest_checks,
|
||||
}
|
||||
manifest_path = output_dir / "evidence.yaml"
|
||||
manifest_path.write_text(
|
||||
yaml.safe_dump(manifest, sort_keys=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"evidence_manifest": str(manifest_path),
|
||||
"profile_sha256": profile_sha256,
|
||||
"checks": len(manifest_checks),
|
||||
},
|
||||
sort_keys=True,
|
||||
)
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -0,0 +1,487 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Qualify Gazebo pause, single-step, speed and resource behavior on SIM S0."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import asdict, dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
TIME_SCHEMA = "missioncore.simulation-time-control/v1"
|
||||
RESOURCE_SCHEMA = "missioncore.simulation-resource-baseline/v1"
|
||||
MIB = 1024**2
|
||||
GIB = 1024**3
|
||||
|
||||
|
||||
class ProbeError(RuntimeError):
|
||||
"""A runtime observation could not satisfy the reviewed S0 contract."""
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class WorldStats:
|
||||
sim_time_ns: int
|
||||
real_time_ns: int
|
||||
iterations: int
|
||||
paused: bool
|
||||
real_time_factor: float
|
||||
step_size_ns: int
|
||||
|
||||
|
||||
def _run(command: list[str], *, timeout: float) -> subprocess.CompletedProcess[str]:
|
||||
result = subprocess.run(
|
||||
command,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=timeout,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise ProbeError(
|
||||
f"command failed ({result.returncode}): {' '.join(command)}; "
|
||||
f"stderr={result.stderr.strip()!r}"
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def _time_field(message: str, field: str) -> int:
|
||||
match = re.search(rf"\b{re.escape(field)}\s*\{{(?P<body>.*?)\}}", message, re.DOTALL)
|
||||
if match is None:
|
||||
raise ProbeError(f"Gazebo stats omitted {field}")
|
||||
body = match.group("body")
|
||||
seconds = re.search(r"\bsec:\s*(-?\d+)", body)
|
||||
nanoseconds = re.search(r"\bnsec:\s*(-?\d+)", body)
|
||||
sec_value = int(seconds.group(1)) if seconds is not None else 0
|
||||
nsec_value = int(nanoseconds.group(1)) if nanoseconds is not None else 0
|
||||
return sec_value * 1_000_000_000 + nsec_value
|
||||
|
||||
|
||||
def _scalar_field(
|
||||
message: str,
|
||||
field: str,
|
||||
converter: type[int] | type[float],
|
||||
*,
|
||||
default: int | float | None = None,
|
||||
) -> int | float:
|
||||
match = re.search(rf"\b{re.escape(field)}:\s*([^\s]+)", message)
|
||||
if match is None:
|
||||
if default is None:
|
||||
raise ProbeError(f"Gazebo stats omitted {field}")
|
||||
return default
|
||||
return converter(match.group(1))
|
||||
|
||||
|
||||
def _stats_snapshot() -> WorldStats:
|
||||
result = _run(
|
||||
["gz", "topic", "--echo", "--topic", "/stats", "-n", "1"],
|
||||
timeout=8,
|
||||
)
|
||||
paused_match = re.search(r"\bpaused:\s*(true|false)", result.stdout)
|
||||
paused = paused_match is not None and paused_match.group(1) == "true"
|
||||
return WorldStats(
|
||||
sim_time_ns=_time_field(result.stdout, "sim_time"),
|
||||
real_time_ns=_time_field(result.stdout, "real_time"),
|
||||
iterations=int(_scalar_field(result.stdout, "iterations", int)),
|
||||
paused=paused,
|
||||
real_time_factor=float(
|
||||
_scalar_field(result.stdout, "real_time_factor", float, default=0.0)
|
||||
),
|
||||
step_size_ns=_time_field(result.stdout, "step_size"),
|
||||
)
|
||||
|
||||
|
||||
def _control(world: str, request: str) -> str:
|
||||
result = _run(
|
||||
[
|
||||
"gz",
|
||||
"service",
|
||||
"-s",
|
||||
f"/world/{world}/control",
|
||||
"--reqtype",
|
||||
"gz.msgs.WorldControl",
|
||||
"--reptype",
|
||||
"gz.msgs.Boolean",
|
||||
"--timeout",
|
||||
"5000",
|
||||
"--req",
|
||||
request,
|
||||
],
|
||||
timeout=8,
|
||||
)
|
||||
if "data: true" not in result.stdout:
|
||||
raise ProbeError(
|
||||
f"Gazebo rejected world control request {request!r}: {result.stdout.strip()!r}"
|
||||
)
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def _wait_for_pause_state(expected: bool, *, timeout: float = 6) -> WorldStats:
|
||||
deadline = time.monotonic() + timeout
|
||||
observed: WorldStats | None = None
|
||||
while time.monotonic() < deadline:
|
||||
observed = _stats_snapshot()
|
||||
if observed.paused is expected:
|
||||
return observed
|
||||
raise ProbeError(f"Gazebo pause state did not become {expected}; last={observed!r}")
|
||||
|
||||
|
||||
def _wait_for_iteration(minimum: int, *, timeout: float = 6) -> WorldStats:
|
||||
deadline = time.monotonic() + timeout
|
||||
observed: WorldStats | None = None
|
||||
while time.monotonic() < deadline:
|
||||
observed = _stats_snapshot()
|
||||
if observed.iterations >= minimum:
|
||||
return observed
|
||||
raise ProbeError(f"Gazebo iteration did not reach {minimum}; last={observed!r}")
|
||||
|
||||
|
||||
def _meminfo() -> dict[str, int]:
|
||||
values: dict[str, int] = {}
|
||||
for line in Path("/proc/meminfo").read_text(encoding="utf-8").splitlines():
|
||||
key, raw = line.split(":", maxsplit=1)
|
||||
amount = int(raw.strip().split()[0])
|
||||
values[key] = amount * 1024
|
||||
return values
|
||||
|
||||
|
||||
def _owned_processes(
|
||||
*,
|
||||
pids: set[int],
|
||||
process_groups: set[int],
|
||||
) -> list[dict[str, int | float | str]]:
|
||||
result = _run(
|
||||
[
|
||||
"ps",
|
||||
"-eo",
|
||||
"pid=,ppid=,pgid=,rss=,pcpu=,comm=",
|
||||
],
|
||||
timeout=5,
|
||||
)
|
||||
processes: list[dict[str, int | float | str]] = []
|
||||
for line in result.stdout.splitlines():
|
||||
fields = line.split(maxsplit=5)
|
||||
if len(fields) != 6:
|
||||
continue
|
||||
pid, ppid, pgid, rss_kib, cpu_percent, command = fields
|
||||
pid_value = int(pid)
|
||||
pgid_value = int(pgid)
|
||||
if pid_value not in pids and pgid_value not in process_groups:
|
||||
continue
|
||||
processes.append(
|
||||
{
|
||||
"pid": pid_value,
|
||||
"ppid": int(ppid),
|
||||
"pgid": pgid_value,
|
||||
"rss_bytes": int(rss_kib) * 1024,
|
||||
"cpu_percent": float(cpu_percent),
|
||||
"command": command,
|
||||
}
|
||||
)
|
||||
return processes
|
||||
|
||||
|
||||
def _gpu_sample() -> dict[str, int] | None:
|
||||
if shutil.which("nvidia-smi") is None:
|
||||
return None
|
||||
result = _run(
|
||||
[
|
||||
"nvidia-smi",
|
||||
"--query-gpu=memory.used,utilization.gpu",
|
||||
"--format=csv,noheader,nounits",
|
||||
],
|
||||
timeout=5,
|
||||
)
|
||||
first_line = result.stdout.strip().splitlines()[0]
|
||||
memory_mib, utilization_percent = (int(value.strip()) for value in first_line.split(","))
|
||||
return {
|
||||
"global_memory_used_mib": memory_mib,
|
||||
"global_utilization_percent": utilization_percent,
|
||||
}
|
||||
|
||||
|
||||
class ResourceSampler:
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
owned_pids: set[int],
|
||||
owned_process_groups: set[int],
|
||||
storage_root: Path,
|
||||
interval_seconds: float,
|
||||
) -> None:
|
||||
self._owned_pids = owned_pids
|
||||
self._owned_process_groups = owned_process_groups
|
||||
self._storage_root = storage_root
|
||||
self._interval_seconds = interval_seconds
|
||||
self._stop = threading.Event()
|
||||
self._thread = threading.Thread(target=self._run, name="s0-resource-sampler")
|
||||
self.samples: list[dict[str, Any]] = []
|
||||
self.error: str | None = None
|
||||
|
||||
def start(self) -> None:
|
||||
self._thread.start()
|
||||
|
||||
def stop(self) -> None:
|
||||
self._stop.set()
|
||||
self._thread.join(timeout=max(5.0, self._interval_seconds * 4))
|
||||
if self._thread.is_alive():
|
||||
raise ProbeError("resource sampler did not stop")
|
||||
if self.error is not None:
|
||||
raise ProbeError(f"resource sampler failed: {self.error}")
|
||||
|
||||
def _run(self) -> None:
|
||||
try:
|
||||
while True:
|
||||
processes = _owned_processes(
|
||||
pids=self._owned_pids,
|
||||
process_groups=self._owned_process_groups,
|
||||
)
|
||||
memory = _meminfo()
|
||||
disk = shutil.disk_usage(self._storage_root)
|
||||
self.samples.append(
|
||||
{
|
||||
"monotonic_ns": time.monotonic_ns(),
|
||||
"owned_rss_bytes": sum(int(process["rss_bytes"]) for process in processes),
|
||||
"owned_cpu_percent": sum(
|
||||
float(process["cpu_percent"]) for process in processes
|
||||
),
|
||||
"memory_available_bytes": memory["MemAvailable"],
|
||||
"swap_used_bytes": memory["SwapTotal"] - memory["SwapFree"],
|
||||
"disk_free_bytes": disk.free,
|
||||
"gpu": _gpu_sample(),
|
||||
"processes": processes,
|
||||
}
|
||||
)
|
||||
if self._stop.wait(self._interval_seconds):
|
||||
break
|
||||
except Exception as error: # noqa: BLE001 - preserve factual sampler failure
|
||||
self.error = f"{type(error).__name__}: {error}"
|
||||
|
||||
|
||||
def _load_contract(profile_path: Path, speed_factor: int) -> dict[str, int | str]:
|
||||
profile = yaml.safe_load(profile_path.read_text(encoding="utf-8"))
|
||||
if not isinstance(profile, dict):
|
||||
raise ProbeError("qualification profile must be a mapping")
|
||||
runtime = profile.get("runtime_acceptance")
|
||||
storage = profile.get("storage")
|
||||
if not isinstance(runtime, dict) or not isinstance(storage, dict):
|
||||
raise ProbeError("qualification profile omitted runtime_acceptance or storage")
|
||||
factors = runtime.get("speed_factors")
|
||||
if not isinstance(factors, list) or speed_factor not in factors:
|
||||
raise ProbeError(f"speed factor {speed_factor} is not declared by the profile")
|
||||
return {
|
||||
"world": str(runtime["world"]),
|
||||
"model": str(runtime["model"]),
|
||||
"physics_step_ns": int(runtime["physics_step_ns"]),
|
||||
"pause_max_advance_ns": int(runtime["pause_max_advance_ns"]),
|
||||
"rtf_tolerance_percent": int(runtime["rtf_tolerance_percent"]),
|
||||
"measurement_window_seconds": int(runtime["measurement_window_seconds"]),
|
||||
"resource_sample_interval_milliseconds": int(
|
||||
runtime["resource_sample_interval_milliseconds"]
|
||||
),
|
||||
"max_owned_rss_mib": int(runtime["max_owned_rss_mib"]),
|
||||
"max_owned_cpu_percent": int(runtime["max_owned_cpu_percent"]),
|
||||
"minimum_available_memory_gib": int(runtime["minimum_available_memory_gib"]),
|
||||
"stop_below_gib": int(storage["stop_below_gib"]),
|
||||
}
|
||||
|
||||
|
||||
def _resource_report(
|
||||
*,
|
||||
samples: list[dict[str, Any]],
|
||||
contract: dict[str, int | str],
|
||||
speed_factor: int,
|
||||
) -> dict[str, Any]:
|
||||
if not samples:
|
||||
raise ProbeError("resource sampler produced no samples")
|
||||
peak_rss = max(int(sample["owned_rss_bytes"]) for sample in samples)
|
||||
peak_cpu = max(float(sample["owned_cpu_percent"]) for sample in samples)
|
||||
minimum_memory = min(int(sample["memory_available_bytes"]) for sample in samples)
|
||||
minimum_disk = min(int(sample["disk_free_bytes"]) for sample in samples)
|
||||
maximum_swap = max(int(sample["swap_used_bytes"]) for sample in samples)
|
||||
gpu_samples = [sample["gpu"] for sample in samples if sample["gpu"] is not None]
|
||||
checks = {
|
||||
"sample_count": len(samples) >= 6,
|
||||
"owned_rss": peak_rss <= int(contract["max_owned_rss_mib"]) * MIB,
|
||||
"owned_cpu": peak_cpu <= int(contract["max_owned_cpu_percent"]),
|
||||
"available_memory": (minimum_memory >= int(contract["minimum_available_memory_gib"]) * GIB),
|
||||
"disk_floor": minimum_disk >= int(contract["stop_below_gib"]) * GIB,
|
||||
}
|
||||
summary: dict[str, Any] = {
|
||||
"sample_count": len(samples),
|
||||
"peak_owned_rss_bytes": peak_rss,
|
||||
"peak_owned_cpu_percent": peak_cpu,
|
||||
"minimum_memory_available_bytes": minimum_memory,
|
||||
"minimum_disk_free_bytes": minimum_disk,
|
||||
"maximum_swap_used_bytes": maximum_swap,
|
||||
"gpu_metrics_are_global_and_include_co_tenants": True,
|
||||
}
|
||||
if gpu_samples:
|
||||
summary["peak_global_gpu_memory_used_mib"] = max(
|
||||
int(sample["global_memory_used_mib"]) for sample in gpu_samples
|
||||
)
|
||||
summary["peak_global_gpu_utilization_percent"] = max(
|
||||
int(sample["global_utilization_percent"]) for sample in gpu_samples
|
||||
)
|
||||
return {
|
||||
"schema_version": RESOURCE_SCHEMA,
|
||||
"speed_factor": speed_factor,
|
||||
"actuator_authority": False,
|
||||
"checks": checks,
|
||||
"summary": summary,
|
||||
"samples": samples,
|
||||
"verdict": "pass" if all(checks.values()) else "fail",
|
||||
}
|
||||
|
||||
|
||||
def _write_json(path: Path, payload: dict[str, Any]) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def _arguments() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--profile", type=Path, required=True)
|
||||
parser.add_argument("--speed-factor", type=int, required=True)
|
||||
parser.add_argument("--time-output", type=Path, required=True)
|
||||
parser.add_argument("--resource-output", type=Path, required=True)
|
||||
parser.add_argument("--storage-root", type=Path, required=True)
|
||||
parser.add_argument("--owned-pid", action="append", type=int, default=[])
|
||||
parser.add_argument("--owned-pgid", action="append", type=int, default=[])
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
args = _arguments()
|
||||
contract = _load_contract(args.profile, args.speed_factor)
|
||||
sampler = ResourceSampler(
|
||||
owned_pids=set(args.owned_pid),
|
||||
owned_process_groups=set(args.owned_pgid),
|
||||
storage_root=args.storage_root,
|
||||
interval_seconds=int(contract["resource_sample_interval_milliseconds"]) / 1000,
|
||||
)
|
||||
time_report: dict[str, Any] = {
|
||||
"schema_version": TIME_SCHEMA,
|
||||
"speed_factor": args.speed_factor,
|
||||
"world": contract["world"],
|
||||
"model": contract["model"],
|
||||
"actuator_authority": False,
|
||||
"contract": contract,
|
||||
}
|
||||
exit_code = 1
|
||||
sampler.start()
|
||||
try:
|
||||
world = str(contract["world"])
|
||||
_control(world, "pause: true")
|
||||
paused_start = _wait_for_pause_state(True)
|
||||
time.sleep(1.0)
|
||||
paused_end = _stats_snapshot()
|
||||
pause_delta = paused_end.sim_time_ns - paused_start.sim_time_ns
|
||||
pause_pass = paused_end.paused and 0 <= pause_delta <= int(contract["pause_max_advance_ns"])
|
||||
|
||||
step_before = paused_end
|
||||
_control(world, "pause: true, multi_step: 1")
|
||||
step_after = _wait_for_iteration(step_before.iterations + 1)
|
||||
step_delta = step_after.sim_time_ns - step_before.sim_time_ns
|
||||
iteration_delta = step_after.iterations - step_before.iterations
|
||||
step_pass = (
|
||||
step_after.paused
|
||||
and step_delta == int(contract["physics_step_ns"])
|
||||
and iteration_delta == 1
|
||||
)
|
||||
|
||||
_control(world, "pause: false")
|
||||
_wait_for_pause_state(False)
|
||||
time.sleep(0.75)
|
||||
speed_start = _stats_snapshot()
|
||||
time.sleep(int(contract["measurement_window_seconds"]))
|
||||
speed_end = _stats_snapshot()
|
||||
sim_delta = speed_end.sim_time_ns - speed_start.sim_time_ns
|
||||
real_delta = speed_end.real_time_ns - speed_start.real_time_ns
|
||||
if real_delta <= 0:
|
||||
raise ProbeError("Gazebo real-time delta was not positive")
|
||||
measured_rtf = sim_delta / real_delta
|
||||
tolerance = int(contract["rtf_tolerance_percent"]) / 100
|
||||
lower = args.speed_factor * (1 - tolerance)
|
||||
upper = args.speed_factor * (1 + tolerance)
|
||||
speed_pass = lower <= measured_rtf <= upper
|
||||
|
||||
time_report.update(
|
||||
{
|
||||
"pause": {
|
||||
"before": asdict(paused_start),
|
||||
"after": asdict(paused_end),
|
||||
"observed_advance_ns": pause_delta,
|
||||
"pass": pause_pass,
|
||||
},
|
||||
"single_step": {
|
||||
"before": asdict(step_before),
|
||||
"after": asdict(step_after),
|
||||
"observed_advance_ns": step_delta,
|
||||
"observed_iteration_delta": iteration_delta,
|
||||
"pass": step_pass,
|
||||
},
|
||||
"speed": {
|
||||
"before": asdict(speed_start),
|
||||
"after": asdict(speed_end),
|
||||
"simulation_advance_ns": sim_delta,
|
||||
"real_advance_ns": real_delta,
|
||||
"measured_rtf": measured_rtf,
|
||||
"accepted_range": [lower, upper],
|
||||
"pass": speed_pass,
|
||||
},
|
||||
}
|
||||
)
|
||||
time_report["verdict"] = "pass" if pause_pass and step_pass and speed_pass else "fail"
|
||||
except Exception as error: # noqa: BLE001 - persist the factual probe failure
|
||||
time_report["verdict"] = "fail"
|
||||
time_report["error"] = f"{type(error).__name__}: {error}"
|
||||
finally:
|
||||
try:
|
||||
_control(str(contract["world"]), "pause: false")
|
||||
except Exception as resume_error: # noqa: BLE001 - report cleanup failure
|
||||
time_report["resume_error"] = f"{type(resume_error).__name__}: {resume_error}"
|
||||
time_report["verdict"] = "fail"
|
||||
try:
|
||||
sampler.stop()
|
||||
resource_report = _resource_report(
|
||||
samples=sampler.samples,
|
||||
contract=contract,
|
||||
speed_factor=args.speed_factor,
|
||||
)
|
||||
except Exception as resource_error: # noqa: BLE001
|
||||
resource_report = {
|
||||
"schema_version": RESOURCE_SCHEMA,
|
||||
"speed_factor": args.speed_factor,
|
||||
"actuator_authority": False,
|
||||
"verdict": "fail",
|
||||
"error": f"{type(resource_error).__name__}: {resource_error}",
|
||||
"samples": sampler.samples,
|
||||
}
|
||||
_write_json(args.time_output, time_report)
|
||||
_write_json(args.resource_output, resource_report)
|
||||
|
||||
if time_report["verdict"] == "pass" and resource_report["verdict"] == "pass":
|
||||
exit_code = 0
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"time_verdict": time_report["verdict"],
|
||||
"resource_verdict": resource_report["verdict"],
|
||||
"speed_factor": args.speed_factor,
|
||||
},
|
||||
sort_keys=True,
|
||||
)
|
||||
)
|
||||
return exit_code
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -58,6 +58,21 @@ ros:
|
|||
domain_id: 0
|
||||
namespace: /missioncore/polygon/s0
|
||||
|
||||
runtime_acceptance:
|
||||
world: rover
|
||||
model: rover_ackermann_0
|
||||
physics_step_ns: 2000000
|
||||
pause_max_advance_ns: 0
|
||||
speed_factors:
|
||||
- 1
|
||||
- 2
|
||||
rtf_tolerance_percent: 20
|
||||
measurement_window_seconds: 4
|
||||
resource_sample_interval_milliseconds: 500
|
||||
max_owned_rss_mib: 2048
|
||||
max_owned_cpu_percent: 800
|
||||
minimum_available_memory_gib: 8
|
||||
|
||||
components:
|
||||
- id: ros2
|
||||
source_url: https://docs.ros.org/en/jazzy/
|
||||
|
|
@ -65,49 +80,49 @@ components:
|
|||
resolved_version: ros-base=0.11.0-1noble.20260616.084325;ros-gz=1.0.22-1noble.20260616.074726
|
||||
resolved_commit:
|
||||
license: mixed-apache-2.0-bsd
|
||||
qualification_state: candidate
|
||||
qualification_state: accepted
|
||||
- id: gazebo
|
||||
source_url: https://gazebosim.org/docs/harmonic/
|
||||
requested_ref: harmonic
|
||||
resolved_version: gz-harmonic=1.0.0-1~noble;gz-sim=8.14.0-1~noble;sdformat=14.9.0-1~noble
|
||||
resolved_commit:
|
||||
license: apache-2.0
|
||||
qualification_state: candidate
|
||||
qualification_state: accepted
|
||||
- id: px4-autopilot
|
||||
source_url: https://github.com/PX4/PX4-Autopilot
|
||||
requested_ref: v1.17.0
|
||||
resolved_version: v1.17.0
|
||||
resolved_commit: d6f12ad1c4f70ad3230afd7d86e971421e02fef4
|
||||
license: bsd-3-clause
|
||||
qualification_state: candidate
|
||||
qualification_state: accepted
|
||||
- id: px4-msgs
|
||||
source_url: https://github.com/PX4/px4_msgs
|
||||
requested_ref: v1.17.0
|
||||
resolved_version: v1.17.0
|
||||
resolved_commit: 86d8239e962f6939e05c3737784f60c02fa884db
|
||||
license: bsd-3-clause
|
||||
qualification_state: candidate
|
||||
qualification_state: accepted
|
||||
- id: px4-gazebo-models
|
||||
source_url: https://github.com/PX4/PX4-gazebo-models
|
||||
requested_ref: PX4-Autopilot/v1.17.0-submodule
|
||||
resolved_version: px4-v1.17.0-submodule
|
||||
resolved_commit: b6127f4ec20de867e215fb5f78ae88b80f371909
|
||||
license: bsd-3-clause
|
||||
qualification_state: candidate
|
||||
qualification_state: accepted
|
||||
- id: micro-xrce-dds-agent
|
||||
source_url: https://github.com/eProsima/Micro-XRCE-DDS-Agent
|
||||
requested_ref: v2.4.3
|
||||
resolved_version: v2.4.3
|
||||
resolved_commit: 73622810d984349b80bbac0ef55fc0b694d62222
|
||||
license: apache-2.0
|
||||
qualification_state: candidate
|
||||
qualification_state: accepted
|
||||
- id: nav2
|
||||
source_url: https://github.com/ros-navigation/navigation2
|
||||
requested_ref: jazzy@2026-07-24
|
||||
resolved_version: navigation2=1.3.12-1noble.20260615.181551;nav2-bringup=1.3.12-1noble.20260616.082701
|
||||
resolved_commit: c92ea2fd9008f50c0ec8447610800214b2a0dafb
|
||||
license: apache-2.0
|
||||
qualification_state: candidate
|
||||
qualification_state: accepted
|
||||
|
||||
ports:
|
||||
- id: micro-xrce-dds
|
||||
|
|
|
|||
|
|
@ -12,9 +12,17 @@ readonly PX4_ROOT="${MISSIONCORE_SIM_PX4_ROOT:-${D_ROOT}/source/PX4-Autopilot}"
|
|||
readonly AGENT_PREFIX="${MISSIONCORE_SIM_AGENT_PREFIX:-${D_ROOT}/runtime/micro-xrce-dds-agent/2.4.3}"
|
||||
readonly PX4_MSGS_SETUP="${MISSIONCORE_SIM_PX4_MSGS_SETUP:-${D_ROOT}/build/ros2/px4_msgs/install/setup.bash}"
|
||||
readonly ARTIFACT_ROOT="${MISSIONCORE_SIM_ARTIFACT_ROOT:-${D_ROOT}/artifacts/s0}"
|
||||
readonly PROFILE_PATH="${MISSIONCORE_SIM_PROFILE_PATH:-${D_ROOT}/source/qualification-profile.yaml}"
|
||||
readonly TIME_PROBE="${MISSIONCORE_SIM_TIME_PROBE:-${D_ROOT}/source/gazebo_time_control.py}"
|
||||
readonly RUN_ID="${1:-$(date -u +%Y%m%dT%H%M%SZ)}"
|
||||
readonly SPEED_FACTOR="${2:-1}"
|
||||
readonly RUN_DIR="${ARTIFACT_ROOT}/${RUN_ID}"
|
||||
|
||||
if [[ "${SPEED_FACTOR}" != "1" && "${SPEED_FACTOR}" != "2" ]]; then
|
||||
echo "error: speed factor must be one of the reviewed values: 1 or 2" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if [[ "${MISSIONCORE_S0_NETNS:-0}" != "1" ]]; then
|
||||
if [[ "${EUID}" -ne 0 ]]; then
|
||||
echo "error: initial worker-smoke invocation must run as root to create the network namespace" >&2
|
||||
|
|
@ -53,6 +61,11 @@ readonly VEHICLE_STATUS_LOG="${RUN_DIR}/vehicle-status.yaml"
|
|||
readonly CLOCK_LOG="${RUN_DIR}/clock.yaml"
|
||||
readonly NETWORK_LOG="${RUN_DIR}/network.txt"
|
||||
readonly RESOURCE_LOG="${RUN_DIR}/resource-snapshot.txt"
|
||||
readonly GZ_SERVICES_LOG="${RUN_DIR}/gazebo-services.txt"
|
||||
readonly GZ_TOPICS_LOG="${RUN_DIR}/gazebo-topics.txt"
|
||||
readonly TIME_CONTROL_LOG="${RUN_DIR}/time-control.json"
|
||||
readonly RESOURCE_BASELINE_LOG="${RUN_DIR}/resource-baseline.json"
|
||||
readonly TIME_PROBE_STDOUT="${RUN_DIR}/time-probe.stdout"
|
||||
readonly RESULT_LOG="${RUN_DIR}/result.env"
|
||||
|
||||
for required_file in \
|
||||
|
|
@ -60,7 +73,9 @@ for required_file in \
|
|||
"${PX4_ROOT}/Tools/simulation/gz/worlds/rover.sdf" \
|
||||
"${AGENT_BIN}" \
|
||||
"/opt/ros/jazzy/setup.bash" \
|
||||
"${PX4_MSGS_SETUP}"; do
|
||||
"${PX4_MSGS_SETUP}" \
|
||||
"${PROFILE_PATH}" \
|
||||
"${TIME_PROBE}"; do
|
||||
if [[ ! -e "${required_file}" ]]; then
|
||||
echo "error: required S0 runtime input is missing: ${required_file}" >&2
|
||||
exit 2
|
||||
|
|
@ -134,7 +149,7 @@ fi
|
|||
} >"${NETWORK_LOG}"
|
||||
|
||||
coproc PX4_PROCESS {
|
||||
exec setsid env HEADLESS=1 make -C "${PX4_ROOT}" \
|
||||
exec setsid env HEADLESS=1 PX4_SIM_SPEED_FACTOR="${SPEED_FACTOR}" make -C "${PX4_ROOT}" \
|
||||
px4_sitl gz_rover_ackermann >"${PX4_LOG}" 2>&1
|
||||
}
|
||||
px4_pgid="${PX4_PROCESS_PID}"
|
||||
|
|
@ -168,6 +183,8 @@ fi
|
|||
echo "runtime_listeners=loopback-only-netns"
|
||||
ss -lunp
|
||||
} >>"${NETWORK_LOG}"
|
||||
gz service -l >"${GZ_SERVICES_LOG}"
|
||||
gz topic -l >"${GZ_TOPICS_LOG}"
|
||||
|
||||
set +u
|
||||
# shellcheck source=/dev/null
|
||||
|
|
@ -189,10 +206,20 @@ if grep -Fq "/fmu/out/vehicle_status_v1" "${TOPICS_LOG}"; then
|
|||
vehicle_status_topic="/fmu/out/vehicle_status_v1"
|
||||
fi
|
||||
timeout 20s ros2 topic echo --once \
|
||||
"${vehicle_status_topic}" >"${VEHICLE_STATUS_LOG}"
|
||||
"${vehicle_status_topic}" px4_msgs/msg/VehicleStatus >"${VEHICLE_STATUS_LOG}"
|
||||
timeout 20s ros2 topic echo --once \
|
||||
/clock rosgraph_msgs/msg/Clock >"${CLOCK_LOG}"
|
||||
|
||||
python3 "${TIME_PROBE}" \
|
||||
--profile "${PROFILE_PATH}" \
|
||||
--speed-factor "${SPEED_FACTOR}" \
|
||||
--time-output "${TIME_CONTROL_LOG}" \
|
||||
--resource-output "${RESOURCE_BASELINE_LOG}" \
|
||||
--storage-root "${D_ROOT}" \
|
||||
--owned-pid "${agent_pid}" \
|
||||
--owned-pgid "${px4_pgid}" \
|
||||
--owned-pgid "${bridge_pgid}" >"${TIME_PROBE_STDOUT}"
|
||||
|
||||
{
|
||||
echo "sample=non-acceptance-point-snapshot"
|
||||
free -b
|
||||
|
|
@ -219,6 +246,8 @@ px4_status="fail"
|
|||
dds_status="fail"
|
||||
vehicle_status="fail"
|
||||
clock_status="fail"
|
||||
time_control_status="fail"
|
||||
resource_baseline_status="fail"
|
||||
lifecycle_status="fail"
|
||||
|
||||
grep -Fq "world: rover, model: rover_ackermann_0" "${PX4_LOG}" && px4_status="pass"
|
||||
|
|
@ -230,6 +259,9 @@ if grep -Fq "session established" "${AGENT_LOG}" \
|
|||
fi
|
||||
grep -Fq "arming_state:" "${VEHICLE_STATUS_LOG}" && vehicle_status="pass"
|
||||
grep -Fq "clock:" "${CLOCK_LOG}" && clock_status="pass"
|
||||
grep -Fq '"verdict": "pass"' "${TIME_CONTROL_LOG}" && time_control_status="pass"
|
||||
grep -Fq '"verdict": "pass"' "${RESOURCE_BASELINE_LOG}" \
|
||||
&& resource_baseline_status="pass"
|
||||
if ! pgrep -f "${PX4_ROOT}/build/px4_sitl_default/bin/px4" >/dev/null \
|
||||
&& ! pgrep -f "gz sim.*rover.sdf" >/dev/null \
|
||||
&& ! pgrep -f "${AGENT_BIN}" >/dev/null; then
|
||||
|
|
@ -240,10 +272,13 @@ fi
|
|||
echo "run_id=${RUN_ID}"
|
||||
echo "network_scope=loopback-only-netns"
|
||||
echo "actuator_authority=false"
|
||||
echo "speed_factor=${SPEED_FACTOR}"
|
||||
echo "px4_stock_rover=${px4_status}"
|
||||
echo "uxrce_dds=${dds_status}"
|
||||
echo "ros2_vehicle_status=${vehicle_status}"
|
||||
echo "gazebo_clock=${clock_status}"
|
||||
echo "pause_step_speed=${time_control_status}"
|
||||
echo "resource_baseline=${resource_baseline_status}"
|
||||
echo "clean_lifecycle=${lifecycle_status}"
|
||||
} >"${RESULT_LOG}"
|
||||
|
||||
|
|
@ -251,6 +286,8 @@ if [[ "${px4_status}" != "pass" \
|
|||
|| "${dds_status}" != "pass" \
|
||||
|| "${vehicle_status}" != "pass" \
|
||||
|| "${clock_status}" != "pass" \
|
||||
|| "${time_control_status}" != "pass" \
|
||||
|| "${resource_baseline_status}" != "pass" \
|
||||
|| "${lifecycle_status}" != "pass" ]]; then
|
||||
echo "error: one or more S0 smoke checks failed; inspect ${RUN_DIR}" >&2
|
||||
exit 1
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@
|
|||
from k1link.simulation.s0 import (
|
||||
CheckStatus,
|
||||
DoctorVerdict,
|
||||
RuntimeAcceptance,
|
||||
S0DoctorReport,
|
||||
S0Profile,
|
||||
S0ProfileError,
|
||||
|
|
@ -13,6 +14,7 @@ from k1link.simulation.s0 import (
|
|||
__all__ = [
|
||||
"CheckStatus",
|
||||
"DoctorVerdict",
|
||||
"RuntimeAcceptance",
|
||||
"S0DoctorReport",
|
||||
"S0Profile",
|
||||
"S0ProfileError",
|
||||
|
|
|
|||
|
|
@ -90,6 +90,21 @@ class RosProfile:
|
|||
namespace: str
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RuntimeAcceptance:
|
||||
world: str
|
||||
model: str
|
||||
physics_step_ns: int
|
||||
pause_max_advance_ns: int
|
||||
speed_factors: tuple[int, ...]
|
||||
rtf_tolerance_percent: int
|
||||
measurement_window_seconds: int
|
||||
resource_sample_interval_milliseconds: int
|
||||
max_owned_rss_mib: int
|
||||
max_owned_cpu_percent: int
|
||||
minimum_available_memory_gib: int
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ComponentPin:
|
||||
identifier: str
|
||||
|
|
@ -126,6 +141,7 @@ class S0Profile:
|
|||
clock: ClockPolicy
|
||||
authority: AuthorityPolicy
|
||||
ros: RosProfile
|
||||
runtime_acceptance: RuntimeAcceptance
|
||||
components: tuple[ComponentPin, ...]
|
||||
ports: tuple[PortBinding, ...]
|
||||
processes: tuple[ProcessOwnership, ...]
|
||||
|
|
@ -191,6 +207,7 @@ def load_s0_profile(path: Path) -> S0Profile:
|
|||
"clock",
|
||||
"authority",
|
||||
"ros",
|
||||
"runtime_acceptance",
|
||||
"components",
|
||||
"ports",
|
||||
"processes",
|
||||
|
|
@ -208,6 +225,7 @@ def load_s0_profile(path: Path) -> S0Profile:
|
|||
clock = _parse_clock(document["clock"])
|
||||
authority = _parse_authority(document["authority"])
|
||||
ros = _parse_ros(document["ros"])
|
||||
runtime_acceptance = _parse_runtime_acceptance(document["runtime_acceptance"])
|
||||
components = _parse_components(document["components"])
|
||||
ports = _parse_ports(document["ports"])
|
||||
processes = _parse_processes(document["processes"])
|
||||
|
|
@ -229,6 +247,7 @@ def load_s0_profile(path: Path) -> S0Profile:
|
|||
clock=clock,
|
||||
authority=authority,
|
||||
ros=ros,
|
||||
runtime_acceptance=runtime_acceptance,
|
||||
components=components,
|
||||
ports=ports,
|
||||
processes=processes,
|
||||
|
|
@ -523,6 +542,94 @@ def _parse_ros(value: object) -> RosProfile:
|
|||
return RosProfile(domain_id, namespace)
|
||||
|
||||
|
||||
def _parse_runtime_acceptance(value: object) -> RuntimeAcceptance:
|
||||
data = _mapping(value, "runtime_acceptance")
|
||||
_expect_keys(
|
||||
data,
|
||||
required={
|
||||
"world",
|
||||
"model",
|
||||
"physics_step_ns",
|
||||
"pause_max_advance_ns",
|
||||
"speed_factors",
|
||||
"rtf_tolerance_percent",
|
||||
"measurement_window_seconds",
|
||||
"resource_sample_interval_milliseconds",
|
||||
"max_owned_rss_mib",
|
||||
"max_owned_cpu_percent",
|
||||
"minimum_available_memory_gib",
|
||||
},
|
||||
context="runtime_acceptance",
|
||||
)
|
||||
speed_factors = tuple(
|
||||
_integer(item, f"runtime_acceptance.speed_factors[{index}]")
|
||||
for index, item in enumerate(
|
||||
_sequence(
|
||||
data["speed_factors"],
|
||||
"runtime_acceptance.speed_factors",
|
||||
)
|
||||
)
|
||||
)
|
||||
acceptance = RuntimeAcceptance(
|
||||
world=_safe_id(data["world"], "runtime_acceptance.world"),
|
||||
model=_string(data["model"], "runtime_acceptance.model"),
|
||||
physics_step_ns=_integer(
|
||||
data["physics_step_ns"],
|
||||
"runtime_acceptance.physics_step_ns",
|
||||
),
|
||||
pause_max_advance_ns=_integer(
|
||||
data["pause_max_advance_ns"],
|
||||
"runtime_acceptance.pause_max_advance_ns",
|
||||
),
|
||||
speed_factors=speed_factors,
|
||||
rtf_tolerance_percent=_integer(
|
||||
data["rtf_tolerance_percent"],
|
||||
"runtime_acceptance.rtf_tolerance_percent",
|
||||
),
|
||||
measurement_window_seconds=_integer(
|
||||
data["measurement_window_seconds"],
|
||||
"runtime_acceptance.measurement_window_seconds",
|
||||
),
|
||||
resource_sample_interval_milliseconds=_integer(
|
||||
data["resource_sample_interval_milliseconds"],
|
||||
"runtime_acceptance.resource_sample_interval_milliseconds",
|
||||
),
|
||||
max_owned_rss_mib=_integer(
|
||||
data["max_owned_rss_mib"],
|
||||
"runtime_acceptance.max_owned_rss_mib",
|
||||
),
|
||||
max_owned_cpu_percent=_integer(
|
||||
data["max_owned_cpu_percent"],
|
||||
"runtime_acceptance.max_owned_cpu_percent",
|
||||
),
|
||||
minimum_available_memory_gib=_integer(
|
||||
data["minimum_available_memory_gib"],
|
||||
"runtime_acceptance.minimum_available_memory_gib",
|
||||
),
|
||||
)
|
||||
if acceptance.world != "rover" or acceptance.model != "rover_ackermann_0":
|
||||
raise S0ProfileError("S0 runtime acceptance must remain on the stock Ackermann rover")
|
||||
if acceptance.physics_step_ns != 2_000_000:
|
||||
raise S0ProfileError("S0 physics step must match the pinned rover world")
|
||||
if acceptance.pause_max_advance_ns != 0:
|
||||
raise S0ProfileError("S0 pause acceptance must require a frozen simulation clock")
|
||||
if acceptance.speed_factors != (1, 2):
|
||||
raise S0ProfileError("S0 must qualify exactly the 1x and 2x speed factors")
|
||||
if not 1 <= acceptance.rtf_tolerance_percent <= 30:
|
||||
raise S0ProfileError("S0 RTF tolerance must be within 1..30 percent")
|
||||
if not 3 <= acceptance.measurement_window_seconds <= 30:
|
||||
raise S0ProfileError("S0 RTF measurement window must be within 3..30 seconds")
|
||||
if not 100 <= acceptance.resource_sample_interval_milliseconds <= 2_000:
|
||||
raise S0ProfileError("S0 resource sample interval must be within 100..2000 ms")
|
||||
if not 256 <= acceptance.max_owned_rss_mib <= 8_192:
|
||||
raise S0ProfileError("S0 owned RSS budget must be within 256..8192 MiB")
|
||||
if not 100 <= acceptance.max_owned_cpu_percent <= 3_200:
|
||||
raise S0ProfileError("S0 owned CPU budget must be within 100..3200 percent")
|
||||
if not 1 <= acceptance.minimum_available_memory_gib <= 64:
|
||||
raise S0ProfileError("S0 available-memory floor must be within 1..64 GiB")
|
||||
return acceptance
|
||||
|
||||
|
||||
def _parse_components(value: object) -> tuple[ComponentPin, ...]:
|
||||
items: list[ComponentPin] = []
|
||||
seen: set[str] = set()
|
||||
|
|
|
|||
|
|
@ -49,6 +49,12 @@ def test_canonical_s0_profile_is_strict_and_safe() -> None:
|
|||
assert profile.authority.navigation_or_safety_accepted is False
|
||||
assert profile.authority.direct_actuator_setpoints_allowed is False
|
||||
assert profile.ros.domain_id == 0
|
||||
assert profile.runtime_acceptance.world == "rover"
|
||||
assert profile.runtime_acceptance.model == "rover_ackermann_0"
|
||||
assert profile.runtime_acceptance.physics_step_ns == 2_000_000
|
||||
assert profile.runtime_acceptance.pause_max_advance_ns == 0
|
||||
assert profile.runtime_acceptance.speed_factors == (1, 2)
|
||||
assert profile.runtime_acceptance.rtf_tolerance_percent == 20
|
||||
assert "docker" not in profile.required_tools
|
||||
assert {item.identifier for item in profile.components} >= {
|
||||
"px4-autopilot",
|
||||
|
|
@ -60,8 +66,7 @@ def test_canonical_s0_profile_is_strict_and_safe() -> None:
|
|||
assert components["px4-gazebo-models"].resolved_commit == (
|
||||
"b6127f4ec20de867e215fb5f78ae88b80f371909"
|
||||
)
|
||||
assert components["ros2"].qualification_state == "candidate"
|
||||
assert components["gazebo"].qualification_state == "candidate"
|
||||
assert {component.qualification_state for component in components.values()} == {"accepted"}
|
||||
micro_xrce = next(item for item in profile.ports if item.identifier == "micro-xrce-dds")
|
||||
assert micro_xrce.host_scope == "loopback-only-netns"
|
||||
assert micro_xrce.bind == "0.0.0.0"
|
||||
|
|
@ -126,6 +131,16 @@ def test_profile_rejects_wildcard_bind_outside_loopback_namespace(
|
|||
load_s0_profile(_write_profile(tmp_path, document))
|
||||
|
||||
|
||||
def test_profile_rejects_runtime_speed_contract_drift(tmp_path: Path) -> None:
|
||||
document = _profile_document()
|
||||
runtime = document["runtime_acceptance"]
|
||||
assert isinstance(runtime, dict)
|
||||
runtime["speed_factors"] = [1, 4]
|
||||
|
||||
with pytest.raises(S0ProfileError, match="exactly the 1x and 2x"):
|
||||
load_s0_profile(_write_profile(tmp_path, document))
|
||||
|
||||
|
||||
def test_doctor_never_claims_go_without_target_evidence(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(
|
||||
"k1link.simulation.s0._target_host_match",
|
||||
|
|
|
|||
Loading…
Reference in New Issue