docs: record Mission Core K1 architecture milestone
This commit is contained in:
DCCONSTRUCTIONS
@@ -0,0 +1,338 @@
|
||||
# ADR 0004: Plugin SDK v0alpha2 and experimental device lifecycle
|
||||
|
||||
- Status: accepted
|
||||
- Date: 2026-07-16
|
||||
- Supersedes: the next-milestone assumptions in ADR 0003; it does not replace
|
||||
the v1alpha1 compatibility boundary
|
||||
|
||||
## Context
|
||||
|
||||
ADR 0003 isolated the first concrete device integration from the Mission Core
|
||||
shell, but the backend still exposed one aggregate runtime and the repository
|
||||
had no executable, vendor-neutral definitions for identity, sessions,
|
||||
operations, acquisitions, streams, or evidence. The verified K1 path also fed
|
||||
vendor MQTT topics and decoder results directly into visualization code.
|
||||
|
||||
The next research stage will use an owner-controlled iPhone and LixelGO to
|
||||
observe additional K1 communication. Before that experiment, Mission Core needs
|
||||
stable local meanings and dependency boundaries so that discovered camera or
|
||||
command mechanics do not spread vendor assumptions through the host.
|
||||
|
||||
The domain is still experimental. Only one K1, firmware `3.0.2`, one direct-LAN
|
||||
topology, point cloud, pose, raw report capture, and operator-manual acquisition
|
||||
have physical evidence. Mutating NODE.DC Platform Ontology now would promote
|
||||
single-device hypotheses into platform-wide meanings before a second device or
|
||||
service proves them.
|
||||
|
||||
## Decision
|
||||
|
||||
Mission Core adopts an independently installable Plugin SDK v0alpha2 and a
|
||||
repository-local experimental vocabulary. It does not modify or depend on
|
||||
NODE.DC Platform Ontology at this stage.
|
||||
|
||||
The executable package lives in:
|
||||
|
||||
```text
|
||||
packages/plugin-sdk/
|
||||
pyproject.toml
|
||||
python/missioncore_plugin_sdk/v0alpha2/
|
||||
```
|
||||
|
||||
It contains closed, deeply immutable Pydantic contracts and deterministic JSON
|
||||
Schema export. Nested mappings and collections are frozen after validation;
|
||||
`model_copy(update=...)` revalidates its update before returning a new contract.
|
||||
The contracts cover:
|
||||
|
||||
- device model and device-instance identity;
|
||||
- transport aliases and execution-node bindings;
|
||||
- device-session state;
|
||||
- operation policy, request, acknowledgement, progress, completion, failure,
|
||||
cancellation and timeout;
|
||||
- point-cloud, pose, image, encoded-video and device-status stream envelopes;
|
||||
- payload and evidence handles, evidence records and raw transport records;
|
||||
- compatibility assessments.
|
||||
|
||||
The experimental semantic definitions and promotion rules live in
|
||||
`docs/domain-model/mission-core-experimental-vocabulary-v0alpha2.md`. Breaking
|
||||
wire or semantic changes create a new explicit SDK module. Existing v0alpha2
|
||||
documents and retained evidence are never silently reinterpreted.
|
||||
|
||||
### Domain distinctions
|
||||
|
||||
These terms are deliberately not aliases:
|
||||
|
||||
| Term | Meaning in this milestone |
|
||||
| --- | --- |
|
||||
| Model | A plugin-owned definition of a supported hardware/product model. It is not physical hardware. |
|
||||
| Device | One enrolled or provisionally identified physical instance. Its identity outlives a connection when a stable basis is available. |
|
||||
| Transport alias | A scoped BLE UUID, IP address, SSID, MQTT identity or USB path. It can locate a device but is not automatically a stable `device_id`. |
|
||||
| Device session | One bounded association between one plugin execution agent and one device instance. Restarting or reconnecting may create a new session. |
|
||||
| Acquisition | One bounded production of requested sensor data, potentially containing multiple channels and sources. |
|
||||
| Operation | One requested action with its own ID, deadline, idempotency boundary, progress and terminal outcome. An acknowledgement is not completion. |
|
||||
| Source/channel | A logical producer or consumable data channel within an acquisition or replay. It is neither the physical device nor the viewer URL. |
|
||||
| Evidence | Immutable raw or derived bytes plus content identity, lineage, retention and redaction metadata. Evidence is not decoded state or presentation state. |
|
||||
|
||||
Enrollment, connectivity and acquisition remain independent state machines. A
|
||||
device can be enrolled while offline, connected while idle, or retain identity
|
||||
after an acquisition failure.
|
||||
|
||||
### Manifest and compatibility profiles
|
||||
|
||||
The manifest catalog accepts both v1alpha1 and v1alpha2 device-plugin manifests.
|
||||
v1alpha1 remains valid for existing plugins and does not gain profile semantics
|
||||
implicitly. A v1alpha2 manifest must link every declared model to at least one
|
||||
plugin-local compatibility profile through `profileId`, `modelId`, and a path
|
||||
constrained to that plugin's directory. v1alpha1 remains exactly one model;
|
||||
v1alpha2 can describe more than one independently profiled model.
|
||||
|
||||
The XGRIDS manifest is now `missioncore.nodedc/v1alpha2`, plugin version `0.2.0`,
|
||||
and links exactly one profile:
|
||||
|
||||
```text
|
||||
xgrids.lixelkity-k1.fw-3.0.2.direct-lan.v1
|
||||
```
|
||||
|
||||
The profile is evidence-scoped to exact firmware `3.0.2`, one direct-LAN
|
||||
topology, and the retained laboratory sources. Its independent flags distinguish
|
||||
`observed`, `decoded`, `replay_verified`, `physical_verified`, and
|
||||
`write_enabled`. Unknown or unattested firmware fails closed. The current
|
||||
runtime cannot read firmware from the device; it activates the limited,
|
||||
read-only profile only after explicit operator attestation of firmware `3.0.2`
|
||||
and direct-LAN topology, records an `operator-attested` basis, and does not
|
||||
present that claim as device-derived proof.
|
||||
|
||||
A profile is descriptive compatibility evidence. Loading a manifest or profile
|
||||
does not grant transport permissions, authorization, or device-write authority.
|
||||
The XGRIDS v1 profile loader rejects every attempted `write_enabled: true` and
|
||||
is executed as a fail-closed plugin-activation gate, not only as a test helper.
|
||||
The separately reviewed BLE Wi-Fi provisioning path remains an explicit legacy
|
||||
operator-confirmed mutation and is not authorized by this profile.
|
||||
|
||||
### Semantic acquisition lifecycle
|
||||
|
||||
The transitional backend now separates operation metadata from acquisition
|
||||
state. It assigns provisional device, device-session, acquisition and operation
|
||||
identifiers and records bounded in-process lifecycle snapshots.
|
||||
|
||||
The current XGRIDS acquisition lifecycle is:
|
||||
|
||||
```text
|
||||
prepare
|
||||
-> prepared
|
||||
-> starting (host receiver is not ready yet)
|
||||
-> awaiting_external_start
|
||||
-> acquiring after the first real point frame
|
||||
-> awaiting_external_stop when graceful stop is requested
|
||||
-> finalizing
|
||||
-> completed
|
||||
```
|
||||
|
||||
Failure, abort and interruption are distinct terminal states. `capture-only`
|
||||
stop may terminate the receiver but must report the physical K1 stop as
|
||||
`unknown`. Graceful stop does not claim success until the operator explicitly
|
||||
confirms the physical action.
|
||||
|
||||
The bounded `OperationJournal` records lifecycle metadata, sequence, revision,
|
||||
declared deadline, idempotency key, terminal result/error and evidence references. It is
|
||||
in memory only, limited to one process and intentionally excludes action inputs
|
||||
and secrets. Deadline persistence and distributed enforcement are not implied.
|
||||
It is not a durable audit log, distributed command journal or recovery mechanism.
|
||||
|
||||
HTTP validation failures on the plugin-scoped action endpoint and deprecated
|
||||
`/api/connect` endpoint return a generic, input-free `422` response. Pydantic
|
||||
diagnostics and rejected request values are not reflected to the client. This
|
||||
is defense in depth for the current in-memory credential path, not a secret
|
||||
vault implementation.
|
||||
|
||||
The v1alpha2 manifest declares semantic acquisition actions alongside legacy
|
||||
`stream.*` compatibility actions. `acquisition.start` and `acquisition.stop`
|
||||
currently mutate host-side lifecycle and capture state only. They prompt and
|
||||
observe the verified physical double-click workflow; they do not publish an
|
||||
MQTT modeling request to the K1.
|
||||
|
||||
### Data-plane boundary
|
||||
|
||||
The implemented local visualization path is:
|
||||
|
||||
```text
|
||||
K1 transport message
|
||||
-> raw-first .k1mqtt evidence capture
|
||||
-> bounded latest-wins preview queue
|
||||
-> explicitly injected K1 vendor normalizer
|
||||
-> transport-neutral DecodedPointCloudView / DecodedPoseView
|
||||
-> Rerun bridge
|
||||
-> local embedded Rerun Web Viewer
|
||||
```
|
||||
|
||||
`VisualizationRuntime` has no default vendor decoder. Its composition owner must
|
||||
inject a `CanonicalNormalizer`; the XGRIDS facade explicitly injects
|
||||
`normalize_k1_message`. The Rerun bridge consumes only decoded local views and
|
||||
contains no MQTT topics, raw payload access, protobuf/LZ4 decoder imports, or
|
||||
K1 branches.
|
||||
|
||||
The `src/k1link/data_plane/Decoded*View` classes are allocation-conscious,
|
||||
in-process consumer projections. They are intentionally not the portable
|
||||
Plugin SDK `PointCloudFrame`, `PoseFrame`, or `DeviceStatusFrame` wire
|
||||
contracts. The current hot path does not yet instantiate SDK stream envelopes,
|
||||
nor does the existing raw capture use the SDK `EvidenceStore` protocol. A later
|
||||
process boundary will map portable SDK envelopes into local consumer views.
|
||||
|
||||
This transitional distinction avoids both vendor/viewer coupling and the false
|
||||
claim that cross-process extraction is already complete.
|
||||
|
||||
### Current K1 safety and evidence status
|
||||
|
||||
The current profile and implementation assert only the following:
|
||||
|
||||
- point cloud and pose are observed, bounded-decoded, replay-verified and
|
||||
physically verified for the exact profile;
|
||||
- device status and heartbeat are observed raw channels but have no semantic
|
||||
decoder;
|
||||
- physical scan start and stop use the verified K1 double-click;
|
||||
- application-level K1 command publishing is disabled;
|
||||
- the statically identified modeling topic and action values remain research
|
||||
evidence only because headers, settings, acknowledgement, error, timeout,
|
||||
rollback and save-completion contracts are unresolved;
|
||||
- device calibration command and sensor-to-vehicle extrinsics are unavailable;
|
||||
- camera transport, endpoint, codec, framing and independent camera channels
|
||||
remain unknown. One negative MQTT observation does not prove absence.
|
||||
|
||||
## Compatibility with v1alpha1
|
||||
|
||||
This milestone is additive:
|
||||
|
||||
- the catalog continues to validate v1alpha1 manifests;
|
||||
- the v1alpha1 `state.read` safety requirement remains;
|
||||
- the established composition root, plugin-scoped dispatcher/events, inactive
|
||||
provider behavior and teardown gate remain in force;
|
||||
- legacy `stream.start-live`, `stream.start-replay`, `stream.stop`, and viewer
|
||||
settings actions remain compatibility shims for the proven UI and runtime;
|
||||
- the XGRIDS backend still runs through the reviewed
|
||||
`transitional-in-process` facade in `src/k1link`.
|
||||
|
||||
SDK v0alpha2 contracts do not imply that every current host endpoint, frontend
|
||||
state object, or compatibility shim already emits those contracts.
|
||||
|
||||
## Deliberately deferred
|
||||
|
||||
The following are not complete and must not be inferred from this ADR:
|
||||
|
||||
- process isolation, crash containment or signed independently deployed plugin
|
||||
bundles;
|
||||
- durable operation journal, restart recovery or distributed idempotency;
|
||||
- secret vault implementation, RBAC, host policy enforcement or command
|
||||
approval workflow;
|
||||
- persistent stable device identity and multi-device/multi-session routing;
|
||||
- multiple concurrent acquisitions or multiple plugin models per runtime;
|
||||
- portable SDK-envelope transport in the live hot path;
|
||||
- SDK `EvidenceStore` integration with the existing `.k1mqtt` recorder;
|
||||
- remote Edge/Control Station split, authenticated WAN relay or fleet
|
||||
orchestration;
|
||||
- automatic K1 acquisition/calibration commands;
|
||||
- camera discovery, decode, synchronization or visualization;
|
||||
- verified sensor timestamps, clock synchronization, coordinate convention,
|
||||
sensor-to-vehicle extrinsics or georeferencing.
|
||||
|
||||
Four contract-compatibility questions are explicitly deferred before portable
|
||||
SDK envelopes may enter the live hot path:
|
||||
|
||||
- `ClockDomain` names UTC, device-monotonic and host-monotonic domains, while
|
||||
the current header stores aware datetimes. A later revision must define a
|
||||
language-neutral monotonic tick/nanosecond representation plus its measured
|
||||
relationship to UTC; an aware datetime must not be used as native monotonic
|
||||
time.
|
||||
- `ReferencedPayload` must distinguish total referenced-object length, segment
|
||||
offset and segment length. Until that invariant is fixed, a non-zero-offset
|
||||
shared-memory segment cannot be assumed to satisfy `PointCloudFrame`'s exact
|
||||
layout-length check.
|
||||
- the coarse SDK `AcquisitionState` vocabulary and the richer transitional
|
||||
facade states require an explicit, tested mapping. Matching `v1alpha2` labels
|
||||
alone do not make their state machines interchangeable.
|
||||
- the generic manifest catalog currently validates profile link integrity,
|
||||
model coverage, path confinement and profile identity only. Evidence and
|
||||
device-specific profile semantics remain the responsibility of the plugin's
|
||||
fail-closed loader. A future generic profile schema must not be claimed until
|
||||
that validation boundary is versioned and tested.
|
||||
|
||||
## Consequences
|
||||
|
||||
New K1 protocol discoveries must enter through the compatibility profile,
|
||||
plugin-owned transport/normalizer code and retained raw evidence. They cannot be
|
||||
added directly to Rerun, Mission Core navigation, generic lifecycle code, or the
|
||||
platform ontology.
|
||||
|
||||
The repository can continue laboratory work without prematurely freezing the
|
||||
platform ontology. A future Edge split can preserve SDK identities, operations,
|
||||
stream envelopes and evidence lineage while replacing the in-process facade and
|
||||
local consumer projection.
|
||||
|
||||
The cost is an explicit temporary adapter boundary: the portable SDK contracts,
|
||||
the in-process lifecycle implementation and the decoded consumer views coexist
|
||||
until extraction and replay parity are proven.
|
||||
|
||||
## Workspace-repeatable locked acceptance checks
|
||||
|
||||
The architecture gate is hardware-free and must pass from the repository root:
|
||||
|
||||
```bash
|
||||
uv sync --frozen --group dev
|
||||
uv run pytest \
|
||||
tests/test_plugin_sdk_v0alpha2_contracts.py \
|
||||
tests/test_plugin_catalog.py \
|
||||
tests/test_xgrids_compatibility_profile.py \
|
||||
tests/test_device_lifecycle.py \
|
||||
tests/test_xgrids_acquisition_lifecycle.py \
|
||||
tests/test_canonical_pipeline.py \
|
||||
tests/test_rerun_bridge.py \
|
||||
tests/test_web_validation_security.py
|
||||
uv run ruff check .
|
||||
uv run mypy
|
||||
uv run mypy --strict packages/plugin-sdk/python/missioncore_plugin_sdk
|
||||
uv run python plugins/xgrids-k1/profile_loader.py
|
||||
uv run python -m missioncore_plugin_sdk.v0alpha2 \
|
||||
> /tmp/missioncore-plugin-sdk-v0alpha2.schemas.json
|
||||
```
|
||||
|
||||
The frontend compatibility gate remains:
|
||||
|
||||
```bash
|
||||
cd apps/control-station
|
||||
npm ci
|
||||
npm run test:unit
|
||||
npm run typecheck
|
||||
npm run build
|
||||
```
|
||||
|
||||
No command above scans Bluetooth, contacts the K1, changes the router, or emits
|
||||
a K1 application command.
|
||||
|
||||
These commands are repeatable in the current workspace, not a standalone
|
||||
release installation. The frontend uses sibling `file:` dependencies from
|
||||
`NODEDC_DESIGN_GUIDELINE`, and `package-lock.json` does not pin that checkout's
|
||||
Git revision or content hash. CI and portable packaging remain blocked until
|
||||
those packages are published or vendored, or the build enforces an immutable
|
||||
donor revision plus content verification. The root Python wheel does not yet
|
||||
bundle plugin manifests, the profile loader, or the built frontend.
|
||||
|
||||
## Next evidence gate
|
||||
|
||||
Communication research begins only after the architecture checks pass and the
|
||||
existing physical regression is repeated on the exact profile:
|
||||
|
||||
1. verify firmware `3.0.2` and the owner-controlled direct-LAN topology;
|
||||
2. prepare one operator-manual acquisition;
|
||||
3. start the K1 by physical double-click;
|
||||
4. retain raw `.k1mqtt` evidence outside Git;
|
||||
5. confirm at least one real point frame and pose data reach the viewer through
|
||||
the injected normalizer with no vendor dependency in Rerun;
|
||||
6. stop by the operator-confirmed physical workflow and record whether device
|
||||
stop was confirmed or remains unknown;
|
||||
7. verify that no MQTT request topic was subscribed to or published and no
|
||||
application command was emitted.
|
||||
|
||||
Only then may a separate read-only LixelGO/iPhone communication-observation
|
||||
experiment begin. Its first output is evidence: endpoints, flows, topics,
|
||||
payload signatures, timing and command/response correlation. It does not enable
|
||||
camera or command execution. Any state-changing publisher or new BLE write
|
||||
requires its own profile revision, exact request/acknowledgement/timeout/error
|
||||
contract, replay fixture, operator gate and physical acceptance.
|
||||
Reference in New Issue
Block a user