docs(perception): record lifecycle ownership and expiry evidence
This commit is contained in:
@@ -181,6 +181,44 @@ entries with exact per-reason totals; it is not the durable terminal ledger.
|
||||
These primitives are NOT the controller supervisor, a cross-process GPU lease,
|
||||
StreamStart fencing or the binary data plane. Those stage-2 boundaries remain open.
|
||||
|
||||
Stage-2 lifecycle increment, 2026-09-02 (`6acf468`, `ac69e3b`):
|
||||
`StreamingLifecycle` now supplies the subprocess-backed profile supervisor using
|
||||
the existing `GraphState` and full `StreamStart` identity. `WorkerLease` uses a
|
||||
stable POSIX file lock on one controller-selected, private directory per Worker,
|
||||
plus an atomic/fsynced ownership record. All managed profile containers must use
|
||||
that SAME directory/volume. Neither the directory nor child commands come from
|
||||
an incoming source/job. This cooperative local fence supplements, not replaces,
|
||||
the backend claim and the trusted inventory of unmanaged GPU clients.
|
||||
|
||||
Open acquires ownership before process spawn/warmup; Ready enables admission.
|
||||
Renewal cannot resurrect expired ownership. Admission, each GPU/CPU lane and
|
||||
result publication/receipt validate run/source/worker/epoch/generation and pinned
|
||||
image/profile/config/calibration/clock identities. A mismatched client is rejected
|
||||
without cancelling the current owner. Each compute lane remains single-owner.
|
||||
The watchdog fences an expired owner even without a new frame and terminates
|
||||
only its dedicated child process groups. Cleanup keeps ownership while callbacks,
|
||||
tracked threads, pending/active payloads or child groups remain. Clean retirement
|
||||
persists `released` before unlocking; a higher generation and new epoch may then
|
||||
activate. A controller crash leaves `active` even when the kernel unlocks, so
|
||||
the next owner stays quarantined. No force/unverified recovery API exists.
|
||||
|
||||
Cross-container contention/clean succession/crash quarantine and full-profile
|
||||
lease expiry were exercised on Worker 006. The final timed expiry trace had no
|
||||
receipt at/after the lease deadline, stop requested after 45.74 ms and retirement
|
||||
after 4109.82 ms. This is bounded evidence, not a hard-real-time scheduler guarantee.
|
||||
The holder has a 2 s lease and receives local pilot heartbeats every 250 ms;
|
||||
those settings and the 50 ms watchdog poll are NOT network/vehicle safety limits.
|
||||
|
||||
Current integration remains explicitly diagnostic: `pilot_lifecycle.py` acts as
|
||||
a local controller, legacy GPU services are quiesced by the authorized launcher,
|
||||
the named volume is an isolated test volume, and the image still has code/model
|
||||
mounts. Mounted-code hashes remain in the experiment manifest; an image digest
|
||||
alone cannot seal those overrides. Backend claims/production-wide canonical root,
|
||||
trusted post-crash resource-release recovery, continuous GPU inventory/envelope
|
||||
enforcement, network authentication/heartbeat and binary live ingress are NOT
|
||||
installed or qualified by this increment. Standalone packaging must remove the
|
||||
developer mounts and retain the same lifecycle boundaries.
|
||||
|
||||
`worker_operating_envelope.py` checks a trusted post-warmup snapshot against
|
||||
preregistered hardware/driver/resource/clock conditions and StreamStart identities,
|
||||
ownership and client inventory. Unknown or expired facts fail readiness. The
|
||||
|
||||
Reference in New Issue
Block a user