NODEDC_MISSION_CORE/docs/adr/0004-plugin-sdk-v0alpha2-and-experimental-device-lifecycle.md

# ADR 0004: Plugin SDK v0alpha2 and experimental device lifecycle

- Status: accepted
- Date: 2026-07-16
- Supersedes: the next-milestone assumptions in ADR 0003; it does not replace
  the v1alpha1 compatibility boundary

## Context

ADR 0003 isolated the first concrete device integration from the Mission Core
shell, but the backend still exposed one aggregate runtime and the repository
had no executable, vendor-neutral definitions for identity, sessions,
operations, acquisitions, streams, or evidence. The verified K1 path also fed
vendor MQTT topics and decoder results directly into visualization code.

The next research stage will use an owner-controlled iPhone and LixelGO to
observe additional K1 communication. Before that experiment, Mission Core needs
stable local meanings and dependency boundaries so that discovered camera or
command mechanics do not spread vendor assumptions through the host.

The domain is still experimental. Only one K1, firmware `3.0.2`, one direct-LAN
topology, point cloud, pose, raw report capture, and operator-manual acquisition
have physical evidence. Mutating NODE.DC Platform Ontology now would promote
single-device hypotheses into platform-wide meanings before a second device or
service proves them.

## Decision

Mission Core adopts an independently installable Plugin SDK v0alpha2 and a
repository-local experimental vocabulary. It does not modify or depend on
NODE.DC Platform Ontology at this stage.

The executable package lives in:

```text
packages/plugin-sdk/
  pyproject.toml
  python/missioncore_plugin_sdk/v0alpha2/
```

It contains closed, deeply immutable Pydantic contracts and deterministic JSON
Schema export. Nested mappings and collections are frozen after validation;
`model_copy(update=...)` revalidates its update before returning a new contract.
The contracts cover:

- device model and device-instance identity;
- transport aliases and execution-node bindings;
- device-session state;
- operation policy, request, acknowledgement, progress, completion, failure,
  cancellation and timeout;
- point-cloud, pose, image, encoded-video and device-status stream envelopes;
- payload and evidence handles, evidence records and raw transport records;
- compatibility assessments.

The experimental semantic definitions and promotion rules live in
`docs/domain-model/mission-core-experimental-vocabulary-v0alpha2.md`. Breaking
wire or semantic changes create a new explicit SDK module. Existing v0alpha2
documents and retained evidence are never silently reinterpreted.

### Domain distinctions

These terms are deliberately not aliases:

| Term | Meaning in this milestone |
| --- | --- |
| Model | A plugin-owned definition of a supported hardware/product model. It is not physical hardware. |
| Device | One enrolled or provisionally identified physical instance. Its identity outlives a connection when a stable basis is available. |
| Transport alias | A scoped BLE UUID, IP address, SSID, MQTT identity or USB path. It can locate a device but is not automatically a stable `device_id`. |
| Device session | One bounded association between one plugin execution agent and one device instance. Restarting or reconnecting may create a new session. |
| Acquisition | One bounded production of requested sensor data, potentially containing multiple channels and sources. |
| Operation | One requested action with its own ID, deadline, idempotency boundary, progress and terminal outcome. An acknowledgement is not completion. |
| Source/channel | A logical producer or consumable data channel within an acquisition or replay. It is neither the physical device nor the viewer URL. |
| Evidence | Immutable raw or derived bytes plus content identity, lineage, retention and redaction metadata. Evidence is not decoded state or presentation state. |

Enrollment, connectivity and acquisition remain independent state machines. A
device can be enrolled while offline, connected while idle, or retain identity
after an acquisition failure.

### Manifest and compatibility profiles

The manifest catalog accepts both v1alpha1 and v1alpha2 device-plugin manifests.
v1alpha1 remains valid for existing plugins and does not gain profile semantics
implicitly. A v1alpha2 manifest must link every declared model to at least one
plugin-local compatibility profile through `profileId`, `modelId`, and a path
constrained to that plugin's directory. v1alpha1 remains exactly one model;
v1alpha2 can describe more than one independently profiled model.

The XGRIDS manifest is now `missioncore.nodedc/v1alpha2`, plugin version `0.2.0`,
and links exactly one profile:

```text
xgrids.lixelkity-k1.fw-3.0.2.direct-lan.v1
```

The profile is evidence-scoped to exact firmware `3.0.2`, one direct-LAN
topology, and the retained laboratory sources. Its independent flags distinguish
`observed`, `decoded`, `replay_verified`, `physical_verified`, and
`write_enabled`. Unknown or unattested firmware fails closed. The current
runtime cannot read firmware from the device; it activates the limited,
read-only profile only after explicit operator attestation of firmware `3.0.2`
and direct-LAN topology, records an `operator-attested` basis, and does not
present that claim as device-derived proof.

A profile is descriptive compatibility evidence. Loading a manifest or profile
does not grant transport permissions, authorization, or device-write authority.
The XGRIDS v1 profile loader rejects every attempted `write_enabled: true` and
is executed as a fail-closed plugin-activation gate, not only as a test helper.
The separately reviewed BLE Wi-Fi provisioning path remains an explicit legacy
operator-confirmed mutation and is not authorized by this profile.

### Semantic acquisition lifecycle

The transitional backend now separates operation metadata from acquisition
state. It assigns provisional device, device-session, acquisition and operation
identifiers and records bounded in-process lifecycle snapshots.

The current XGRIDS acquisition lifecycle is:

```text
prepare
  -> prepared
  -> starting (host receiver is not ready yet)
  -> awaiting_external_start
  -> acquiring after the first real point frame
  -> awaiting_external_stop when graceful stop is requested
  -> finalizing
  -> completed
```

Failure, abort and interruption are distinct terminal states. `capture-only`
stop may terminate the receiver but must report the physical K1 stop as
`unknown`. Graceful stop does not claim success until the operator explicitly
confirms the physical action.

The bounded `OperationJournal` records lifecycle metadata, sequence, revision,
declared deadline, idempotency key, terminal result/error and evidence references. It is
in memory only, limited to one process and intentionally excludes action inputs
and secrets. Deadline persistence and distributed enforcement are not implied.
It is not a durable audit log, distributed command journal or recovery mechanism.

HTTP validation failures on the plugin-scoped action endpoint and deprecated
`/api/connect` endpoint return a generic, input-free `422` response. Pydantic
diagnostics and rejected request values are not reflected to the client. This
is defense in depth for the current in-memory credential path, not a secret
vault implementation.

The v1alpha2 manifest declares semantic acquisition actions alongside legacy
`stream.*` compatibility actions. `acquisition.start` and `acquisition.stop`
currently mutate host-side lifecycle and capture state only. They prompt and
observe the verified physical double-click workflow; they do not publish an
MQTT modeling request to the K1.

### Data-plane boundary

The implemented local visualization path is:

```text
K1 transport message
  -> raw-first .k1mqtt evidence capture
  -> bounded latest-wins preview queue
  -> explicitly injected K1 vendor normalizer
  -> transport-neutral DecodedPointCloudView / DecodedPoseView
  -> Rerun bridge
  -> local embedded Rerun Web Viewer
```

`VisualizationRuntime` has no default vendor decoder. Its composition owner must
inject a `CanonicalNormalizer`; the XGRIDS facade explicitly injects
`normalize_k1_message`. The Rerun bridge consumes only decoded local views and
contains no MQTT topics, raw payload access, protobuf/LZ4 decoder imports, or
K1 branches.

The `src/k1link/data_plane/Decoded*View` classes are allocation-conscious,
in-process consumer projections. They are intentionally not the portable
Plugin SDK `PointCloudFrame`, `PoseFrame`, or `DeviceStatusFrame` wire
contracts. The current hot path does not yet instantiate SDK stream envelopes,
nor does the existing raw capture use the SDK `EvidenceStore` protocol. A later
process boundary will map portable SDK envelopes into local consumer views.

This transitional distinction avoids both vendor/viewer coupling and the false
claim that cross-process extraction is already complete.

### Current K1 safety and evidence status

The current profile and implementation assert only the following:

- point cloud and pose are observed, bounded-decoded, replay-verified and
  physically verified for the exact profile;
- device status and heartbeat are observed raw channels but have no semantic
  decoder;
- physical scan start and stop use the verified K1 double-click;
- application-level K1 command publishing is disabled;
- the statically identified modeling topic and action values remain research
  evidence only because headers, settings, acknowledgement, error, timeout,
  rollback and save-completion contracts are unresolved;
- device calibration command and sensor-to-vehicle extrinsics are unavailable;
- camera transport, endpoint, codec, framing and independent camera channels
  remain unknown. One negative MQTT observation does not prove absence.

## Compatibility with v1alpha1

This milestone is additive:

- the catalog continues to validate v1alpha1 manifests;
- the v1alpha1 `state.read` safety requirement remains;
- the established composition root, plugin-scoped dispatcher/events, inactive
  provider behavior and teardown gate remain in force;
- legacy `stream.start-live`, `stream.start-replay`, `stream.stop`, and viewer
  settings actions remain compatibility shims for the proven UI and runtime;
- the XGRIDS backend still runs through the reviewed
  `transitional-in-process` facade in
  `src/k1link/device_plugins/xgrids_k1`;
- every dispatcher and compatibility-route action now crosses immutable SDK
  `RuntimeActionInvocation` and `RuntimeActionResult` validation.

SDK v0alpha2 runtime-action contracts are active in the backend hot path. This
does not imply that frontend state objects, live spatial frames or stored raw
evidence already use every SDK contract.

## Deliberately deferred

The following are not complete and must not be inferred from this ADR:

- process isolation, crash containment or signed independently deployed plugin
  bundles;
- durable operation journal, restart recovery or distributed idempotency;
- secret vault implementation, RBAC, host policy enforcement or command
  approval workflow;
- persistent stable device identity and multi-device/multi-session routing;
- multiple concurrent acquisitions or multiple plugin models per runtime;
- portable SDK-envelope transport in the live hot path;
- SDK `EvidenceStore` integration with the existing `.k1mqtt` recorder;
- remote Edge/Control Station split, authenticated WAN relay or fleet
  orchestration;
- automatic K1 acquisition/calibration commands;
- camera discovery, decode, synchronization or visualization;
- verified sensor timestamps, clock synchronization, coordinate convention,
  sensor-to-vehicle extrinsics or georeferencing.

Four contract-compatibility questions are explicitly deferred before portable
SDK envelopes may enter the live hot path:

- `ClockDomain` names UTC, device-monotonic and host-monotonic domains, while
  the current header stores aware datetimes. A later revision must define a
  language-neutral monotonic tick/nanosecond representation plus its measured
  relationship to UTC; an aware datetime must not be used as native monotonic
  time.
- `ReferencedPayload` must distinguish total referenced-object length, segment
  offset and segment length. Until that invariant is fixed, a non-zero-offset
  shared-memory segment cannot be assumed to satisfy `PointCloudFrame`'s exact
  layout-length check.
- the coarse SDK `AcquisitionState` vocabulary and the richer transitional
  facade states require an explicit, tested mapping. Matching `v1alpha2` labels
  alone do not make their state machines interchangeable.
- the generic manifest catalog currently validates profile link integrity,
  model coverage, path confinement and profile identity only. Evidence and
  device-specific profile semantics remain the responsibility of the plugin's
  fail-closed loader. A future generic profile schema must not be claimed until
  that validation boundary is versioned and tested.

## Consequences

New K1 protocol discoveries must enter through the compatibility profile,
plugin-owned transport/normalizer code and retained raw evidence. They cannot be
added directly to Rerun, Mission Core navigation, generic lifecycle code, or the
platform ontology.

The repository can continue laboratory work without prematurely freezing the
platform ontology. A future Edge split can preserve SDK identities, operations,
stream envelopes and evidence lineage while replacing the in-process facade and
local consumer projection.

The cost is an explicit temporary adapter boundary: the portable SDK contracts,
the in-process lifecycle implementation and the decoded consumer views coexist
until extraction and replay parity are proven.

## Workspace-repeatable locked acceptance checks

The architecture gate is hardware-free and must pass from the repository root:

```bash
uv sync --frozen --group dev
uv run pytest \
  tests/test_plugin_sdk_v0alpha2_contracts.py \
  tests/test_plugin_catalog.py \
  tests/test_xgrids_compatibility_profile.py \
  tests/test_device_lifecycle.py \
  tests/test_xgrids_acquisition_lifecycle.py \
  tests/test_canonical_pipeline.py \
  tests/test_rerun_bridge.py \
  tests/test_web_validation_security.py
uv run ruff check .
uv run mypy
uv run mypy --strict packages/plugin-sdk/python/missioncore_plugin_sdk
uv run python plugins/xgrids-k1/profile_loader.py
uv run python -m missioncore_plugin_sdk.v0alpha2 \
  > /tmp/missioncore-plugin-sdk-v0alpha2.schemas.json
```

The frontend compatibility gate remains:

```bash
cd apps/control-station
npm ci
npm run test:unit
npm run typecheck
npm run build
```

No command above scans Bluetooth, contacts the K1, changes the router, or emits
a K1 application command.

These commands are repeatable in the current workspace, not a standalone
release installation. The frontend uses sibling `file:` dependencies from
`NODEDC_DESIGN_GUIDELINE`, and `package-lock.json` does not pin that checkout's
Git revision or content hash. CI and portable packaging remain blocked until
those packages are published or vendored, or the build enforces an immutable
donor revision plus content verification. The root Python wheel does not yet
bundle plugin manifests, the profile loader, or the built frontend.

## Next evidence gate

Communication research begins only after the architecture checks pass and the
existing physical regression is repeated on the exact profile:

1. verify firmware `3.0.2` and the owner-controlled direct-LAN topology;
2. prepare one operator-manual acquisition;
3. start the K1 by physical double-click;
4. retain raw `.k1mqtt` evidence outside Git;
5. confirm at least one real point frame and pose data reach the viewer through
   the injected normalizer with no vendor dependency in Rerun;
6. stop by the operator-confirmed physical workflow and record whether device
   stop was confirmed or remains unknown;
7. verify that no MQTT request topic was subscribed to or published and no
   application command was emitted.

Only then may a separate read-only LixelGO/iPhone communication-observation
experiment begin. Its first output is evidence: endpoints, flows, topics,
payload signatures, timing and command/response correlation. It does not enable
camera or command execution. Any state-changing publisher or new BLE write
requires its own profile revision, exact request/acknowledgement/timeout/error
contract, replay fixture, operator gate and physical acceptance.