NODEDC_MISSION_CORE/docs/adr/0004-plugin-sdk-v0alpha2-and-experimental-device-lifecycle.md

ADR 0004: Plugin SDK v0alpha2 and experimental device lifecycle

• Status: accepted
• Date: 2026-07-16
• Supersedes: the next-milestone assumptions in ADR 0003; it does not replace the v1alpha1 compatibility boundary

Context

ADR 0003 isolated the first concrete device integration from the Mission Core shell, but the backend still exposed one aggregate runtime and the repository had no executable, vendor-neutral definitions for identity, sessions, operations, acquisitions, streams, or evidence. The verified K1 path also fed vendor MQTT topics and decoder results directly into visualization code.

The next research stage will use an owner-controlled iPhone and LixelGO to observe additional K1 communication. Before that experiment, Mission Core needs stable local meanings and dependency boundaries so that discovered camera or command mechanics do not spread vendor assumptions through the host.

The domain is still experimental. Only one K1, firmware 3.0.2, one direct-LAN topology, point cloud, pose, raw report capture, and operator-manual acquisition have physical evidence. Mutating NODE.DC Platform Ontology now would promote single-device hypotheses into platform-wide meanings before a second device or service proves them.

Decision

Mission Core adopts an independently installable Plugin SDK v0alpha2 and a repository-local experimental vocabulary. It does not modify or depend on NODE.DC Platform Ontology at this stage.

The executable package lives in:

packages/plugin-sdk/
  pyproject.toml
  python/missioncore_plugin_sdk/v0alpha2/

It contains closed, deeply immutable Pydantic contracts and deterministic JSON Schema export. Nested mappings and collections are frozen after validation; model_copy(update=...) revalidates its update before returning a new contract. The contracts cover:

• device model and device-instance identity;
• transport aliases and execution-node bindings;
• device-session state;
• operation policy, request, acknowledgement, progress, completion, failure, cancellation and timeout;
• point-cloud, pose, image, encoded-video and device-status stream envelopes;
• payload and evidence handles, evidence records and raw transport records;
• compatibility assessments.

The experimental semantic definitions and promotion rules live in docs/domain-model/mission-core-experimental-vocabulary-v0alpha2.md. Breaking wire or semantic changes create a new explicit SDK module. Existing v0alpha2 documents and retained evidence are never silently reinterpreted.

Domain distinctions

These terms are deliberately not aliases:

Term	Meaning in this milestone
Model	A plugin-owned definition of a supported hardware/product model. It is not physical hardware.
Device	One enrolled or provisionally identified physical instance. Its identity outlives a connection when a stable basis is available.
Transport alias	A scoped BLE UUID, IP address, SSID, MQTT identity or USB path. It can locate a device but is not automatically a stable device_id.
Device session	One bounded association between one plugin execution agent and one device instance. Restarting or reconnecting may create a new session.
Acquisition	One bounded production of requested sensor data, potentially containing multiple channels and sources.
Operation	One requested action with its own ID, deadline, idempotency boundary, progress and terminal outcome. An acknowledgement is not completion.
Source/channel	A logical producer or consumable data channel within an acquisition or replay. It is neither the physical device nor the viewer URL.
Evidence	Immutable raw or derived bytes plus content identity, lineage, retention and redaction metadata. Evidence is not decoded state or presentation state.

Enrollment, connectivity and acquisition remain independent state machines. A device can be enrolled while offline, connected while idle, or retain identity after an acquisition failure.

Manifest and compatibility profiles

The manifest catalog accepts both v1alpha1 and v1alpha2 device-plugin manifests. v1alpha1 remains valid for existing plugins and does not gain profile semantics implicitly. A v1alpha2 manifest must link every declared model to at least one plugin-local compatibility profile through profileId, modelId, and a path constrained to that plugin's directory. v1alpha1 remains exactly one model; v1alpha2 can describe more than one independently profiled model.

The XGRIDS manifest is now missioncore.nodedc/v1alpha2, plugin version 0.2.0, and links exactly one profile:

xgrids.lixelkity-k1.fw-3.0.2.direct-lan.v1

The profile is evidence-scoped to exact firmware 3.0.2, one direct-LAN topology, and the retained laboratory sources. Its independent flags distinguish observed, decoded, replay_verified, physical_verified, and write_enabled. Unknown or unattested firmware fails closed. The current runtime cannot read firmware from the device; it activates the limited, read-only profile only after explicit operator attestation of firmware 3.0.2 and direct-LAN topology, records an operator-attested basis, and does not present that claim as device-derived proof.

A profile is descriptive compatibility evidence. Loading a manifest or profile does not grant transport permissions, authorization, or device-write authority. The XGRIDS v1 profile loader rejects every attempted write_enabled: true and is executed as a fail-closed plugin-activation gate, not only as a test helper. The separately reviewed BLE Wi-Fi provisioning path remains an explicit legacy operator-confirmed mutation and is not authorized by this profile.

Semantic acquisition lifecycle

The transitional backend now separates operation metadata from acquisition state. It assigns provisional device, device-session, acquisition and operation identifiers and records bounded in-process lifecycle snapshots.

The current XGRIDS acquisition lifecycle is:

prepare
  -> prepared
  -> starting (host receiver is not ready yet)
  -> awaiting_external_start
  -> acquiring after the first real point frame
  -> awaiting_external_stop when graceful stop is requested
  -> finalizing
  -> completed

Failure, abort and interruption are distinct terminal states. capture-only stop may terminate the receiver but must report the physical K1 stop as unknown. Graceful stop does not claim success until the operator explicitly confirms the physical action.

The bounded OperationJournal records lifecycle metadata, sequence, revision, declared deadline, idempotency key, terminal result/error and evidence references. It is in memory only, limited to one process and intentionally excludes action inputs and secrets. Deadline persistence and distributed enforcement are not implied. It is not a durable audit log, distributed command journal or recovery mechanism.

HTTP validation failures on the plugin-scoped action endpoint and deprecated /api/connect endpoint return a generic, input-free 422 response. Pydantic diagnostics and rejected request values are not reflected to the client. This is defense in depth for the current in-memory credential path, not a secret vault implementation.

The v1alpha2 manifest declares semantic acquisition actions alongside legacy stream.* compatibility actions. acquisition.start and acquisition.stop currently mutate host-side lifecycle and capture state only. They prompt and observe the verified physical double-click workflow; they do not publish an MQTT modeling request to the K1.

Data-plane boundary

The implemented local visualization path is:

K1 transport message
  -> raw-first .k1mqtt evidence capture
  -> bounded latest-wins preview queue
  -> explicitly injected K1 vendor normalizer
  -> transport-neutral DecodedPointCloudView / DecodedPoseView
  -> Rerun bridge
  -> local embedded Rerun Web Viewer

VisualizationRuntime has no default vendor decoder. Its composition owner must inject a CanonicalNormalizer; the XGRIDS facade explicitly injects normalize_k1_message. The Rerun bridge consumes only decoded local views and contains no MQTT topics, raw payload access, protobuf/LZ4 decoder imports, or K1 branches.

The src/k1link/data_plane/Decoded*View classes are allocation-conscious, in-process consumer projections. They are intentionally not the portable Plugin SDK PointCloudFrame, PoseFrame, or DeviceStatusFrame wire contracts. The current hot path does not yet instantiate SDK stream envelopes, nor does the existing raw capture use the SDK EvidenceStore protocol. A later process boundary will map portable SDK envelopes into local consumer views.

This transitional distinction avoids both vendor/viewer coupling and the false claim that cross-process extraction is already complete.

Current K1 safety and evidence status

The current profile and implementation assert only the following:

• point cloud and pose are observed, bounded-decoded, replay-verified and physically verified for the exact profile;
• device status and heartbeat are observed raw channels but have no semantic decoder;
• physical scan start and stop use the verified K1 double-click;
• application-level K1 command publishing is disabled;
• the statically identified modeling topic and action values remain research evidence only because headers, settings, acknowledgement, error, timeout, rollback and save-completion contracts are unresolved;
• device calibration command and sensor-to-vehicle extrinsics are unavailable;
• camera transport, endpoint, codec, framing and independent camera channels remain unknown. One negative MQTT observation does not prove absence.

Compatibility with v1alpha1

This milestone is additive:

• the catalog continues to validate v1alpha1 manifests;
• the v1alpha1 state.read safety requirement remains;
• the established composition root, plugin-scoped dispatcher/events, inactive provider behavior and teardown gate remain in force;
• legacy stream.start-live, stream.start-replay, stream.stop, and viewer settings actions remain compatibility shims for the proven UI and runtime;
• the XGRIDS backend still runs through the reviewed transitional-in-process facade in src/k1link/device_plugins/xgrids_k1;
• every dispatcher and compatibility-route action now crosses immutable SDK RuntimeActionInvocation and RuntimeActionResult validation.

SDK v0alpha2 runtime-action contracts are active in the backend hot path. This does not imply that frontend state objects, live spatial frames or stored raw evidence already use every SDK contract.

ADR 0011 later adds runtime-owned descriptor, handshake and lifecycle-health contracts plus a transport seam around this action path. Its current implementation remains in-process and does not revise the device/acquisition semantics accepted here.

Deliberately deferred

The following are not complete and must not be inferred from this ADR:

• process isolation, crash containment or signed independently deployed plugin bundles;
• durable operation journal, restart recovery or distributed idempotency;
• secret vault implementation, RBAC, host policy enforcement or command approval workflow;
• persistent stable device identity and multi-device/multi-session routing;
• multiple concurrent acquisitions or multiple plugin models per runtime;
• portable SDK-envelope transport in the live hot path;
• SDK EvidenceStore integration with the existing .k1mqtt recorder;
• remote Edge/Control Station split, authenticated WAN relay or fleet orchestration;
• automatic K1 acquisition/calibration commands;
• camera discovery, decode, synchronization or visualization;
• verified sensor timestamps, clock synchronization, coordinate convention, sensor-to-vehicle extrinsics or georeferencing.

Four contract-compatibility questions are explicitly deferred before portable SDK envelopes may enter the live hot path:

• ClockDomain names UTC, device-monotonic and host-monotonic domains, while the current header stores aware datetimes. A later revision must define a language-neutral monotonic tick/nanosecond representation plus its measured relationship to UTC; an aware datetime must not be used as native monotonic time.
• ReferencedPayload must distinguish total referenced-object length, segment offset and segment length. Until that invariant is fixed, a non-zero-offset shared-memory segment cannot be assumed to satisfy PointCloudFrame's exact layout-length check.
• the coarse SDK AcquisitionState vocabulary and the richer transitional facade states require an explicit, tested mapping. Matching v1alpha2 labels alone do not make their state machines interchangeable.
• the generic manifest catalog currently validates profile link integrity, model coverage, path confinement and profile identity only. Evidence and device-specific profile semantics remain the responsibility of the plugin's fail-closed loader. A future generic profile schema must not be claimed until that validation boundary is versioned and tested.

Consequences

New K1 protocol discoveries must enter through the compatibility profile, plugin-owned transport/normalizer code and retained raw evidence. They cannot be added directly to Rerun, Mission Core navigation, generic lifecycle code, or the platform ontology.

The repository can continue laboratory work without prematurely freezing the platform ontology. A future Edge split can preserve SDK identities, operations, stream envelopes and evidence lineage while replacing the in-process facade and local consumer projection.

The cost is an explicit temporary adapter boundary: the portable SDK contracts, the in-process lifecycle implementation and the decoded consumer views coexist until extraction and replay parity are proven.

Workspace-repeatable locked acceptance checks

The architecture gate is hardware-free and must pass from the repository root:

uv sync --frozen --group dev
uv run pytest \
  tests/test_plugin_sdk_v0alpha2_contracts.py \
  tests/test_plugin_catalog.py \
  tests/test_xgrids_compatibility_profile.py \
  tests/test_device_lifecycle.py \
  tests/test_xgrids_acquisition_lifecycle.py \
  tests/test_canonical_pipeline.py \
  tests/test_rerun_bridge.py \
  tests/test_web_validation_security.py
uv run ruff check .
uv run mypy
uv run mypy --strict packages/plugin-sdk/python/missioncore_plugin_sdk
uv run python plugins/xgrids-k1/profile_loader.py
uv run python -m missioncore_plugin_sdk.v0alpha2 \
  > /tmp/missioncore-plugin-sdk-v0alpha2.schemas.json

The frontend compatibility gate remains:

cd apps/control-station
npm ci
npm run test:unit
npm run typecheck
npm run build

No command above scans Bluetooth, contacts the K1, changes the router, or emits a K1 application command.

These commands are repeatable in the current workspace, not a standalone release installation. The frontend uses sibling file: dependencies from NODEDC_DESIGN_GUIDELINE, and package-lock.json does not pin that checkout's Git revision or content hash. CI and portable packaging remain blocked until those packages are published or vendored, or the build enforces an immutable donor revision plus content verification. The root Python wheel does not yet bundle plugin manifests, the profile loader, or the built frontend.

Next evidence gate

Communication research begins only after the architecture checks pass and the existing physical regression is repeated on the exact profile:

1. verify firmware 3.0.2 and the owner-controlled direct-LAN topology;
2. prepare one operator-manual acquisition;
3. start the K1 by physical double-click;
4. retain raw .k1mqtt evidence outside Git;
5. confirm at least one real point frame and pose data reach the viewer through the injected normalizer with no vendor dependency in Rerun;
6. stop by the operator-confirmed physical workflow and record whether device stop was confirmed or remains unknown;
7. verify that no MQTT request topic was subscribed to or published and no application command was emitted.

Only then may a separate read-only LixelGO/iPhone communication-observation experiment begin. Its first output is evidence: endpoints, flows, topics, payload signatures, timing and command/response correlation. It does not enable camera or command execution. Any state-changing publisher or new BLE write requires its own profile revision, exact request/acknowledgement/timeout/error contract, replay fixture, operator gate and physical acceptance.