Plugin developer SDK

The SDK gives you a small starting point to develop a BenchWeave device plugin with AI: generate, implement, test, review, then prepare a release. Use this guide for SDK installation, project generation, testing and packaging. The device developer guide (main repository) covers device evidence, integration requirements, firmware and qualification.

The device is the hardware. The plugin is its Python integration and descriptor. Follow the developer directory convention: plugins/<manufacturer>/<name>/, with src/<python_package>/ inside that project. The name normally identifies the device model, such as fnirsi/dps150.

The current SDK is benchweave-sdk 0.15.0 for Python 3.13+, OTDP 0.2.3 and adapter API 1.1, with registry 0.1.3 schemas. Its source lives in this repository. The main BenchWeave repository mounts it at packages/sdk. Build it separately from the gateway. It supplies structural typing interfaces, offline contract validation, a scripted mock host, bounded async conformance helpers, an external project generator and file-inventory helpers.

1. Install the SDK and generate your project

Install the SDK from PyPI:

# plugin authoring: `new` renders the copier template (issue #347 WS2)
pip install 'benchweave-sdk[scaffold]'
# scaffold-free install (check/conformance/preview only)
pip install benchweave-sdk
# or as an isolated CLI tool
uv tool install benchweave-sdk

For the newest unreleased code, install from the default branch:

pip install git+https://github.com/madeinoz67/benchweave-sdk.git

Full channel list: see the README Installation section.

PyPI is the default install channel: pip install benchweave-sdk or uv pip install benchweave-sdk. Contributors and maintainers can instead build from a checkout of the main BenchWeave repository. Run this from that checkout’s root. packages/sdk is the submodule mount path there, not a directory in this repository:

uv build packages/sdk --out-dir dist/sdk

In your development directory, create an environment. Then install the built wheel from the checkout’s dist/sdk/:

uv venv --python 3.13
source .venv/bin/activate
uv pip install 'dist/sdk/benchweave_sdk-*.whl[scaffold]'
benchweave-sdk new plugins/acme/model100 --package benchweave_acme_model100
cd plugins/acme/model100
uv pip install -e '.[test]'
pytest

Keep the environment that contains the SDK wheel. If you create a new environment inside the project, install the wheel there too. For a repeatable uv lock against a locally built wheel, configure that wheel or your organisation’s package source explicitly.

Run benchweave-sdk doctor from the project to compare the pin with the installed SDK. The command names the recorded version, the installed version, and the reinstall command when they differ. The command works offline and does not fail the run. A stale SDK is a report, not an error.

The generated project also carries .copier-answers.yml at its root: the durable provenance record of which SDK template version produced it (new pins the canonical repository and the matching v<version> template tag — template version == SDK version). The pyproject pin that doctor reads remains a mutable declaration the author may move; the answers file is the record.

Moving a project to a newer template: benchweave-sdk upgrade (from the project root, on a committed and clean git tree) fetches the installed SDK’s released template tag from the canonical repository and applies it — author-owned files (pyproject, README, CLAUDE.md, src/<pkg>/**, tests/**) are preserved byte-for-byte; the managed AI-GUIDE.md moves with the template, and a file you both changed gets conflict markers to resolve and commit. Released template tags only: a development install’s version has no tag and the update refuses with upgrade_tag_missing. Projects scaffolded before the copier port gain provenance with benchweave-sdk adopt — it infers the package name and base version from the generated pyproject.toml (--package / --scaffolded-at override) and refuses a project whose provenance it cannot establish.

Replace acme/model100 with your manufacturer and device or plugin name. The destination controls the directory hierarchy. The --package option controls the Python import package. The SDK creates missing parent directories and refuses to overwrite an existing project. A destination that you reach through a symlinked directory is canonicalized before any write. On macOS, /tmp/... is such a symlink. Messages report the real path.

plugins/
└── acme/
    └── model100/                       # Independent build/test/release root
        ├── pyproject.toml
        ├── README.md
        ├── AI-GUIDE.md
        ├── CLAUDE.md                   # Agent notes; root dev tooling
        ├── AGENTS.md                   # SDK-managed agent guide; CLAUDE.md imports it
        ├── .claude/skills/             # Five SDK-managed workflow skills
        ├── src/benchweave_acme_model100/
        │   ├── __init__.py
        │   ├── adapter.py
        │   ├── protocol.py
        │   ├── descriptor.json
        │   ├── protocol.md
        │   ├── vectors.json
        │   └── skills/                 # Seeded agent skills; ship in the wheel
        │       ├── develop-plugin/SKILL.md
        │       └── drive-device/SKILL.md
        └── tests/test_plugin.py

The root also carries AGENTS.md, the SDK-managed agent guide: CLAUDE.md imports it, and benchweave-sdk upgrade refreshes it together with the five workflow skills under .claude/skills/ (benchweave-plugin-workflow, benchweave-descriptor, benchweave-plugin-ui, benchweave-adapter-testing, benchweave-capture). They are repo-level dev tooling outside src/<python_package>/; they never enter the plugin wheel. The seeded package skills inside src/<python_package>/skills/ are different content and do ship in the wheel. If you edit a managed file, the upgrade writes conflict markers; resolve both sides, then commit. A project made before these files existed adds the @AGENTS.md import line to its CLAUDE.md by hand.

Generate the dependency lock and retain it. Add your chosen licence file. When you develop custom firmware, add firmware/ with its own build and tests. The starter does not create these project-specific files. You can copy the manufacturer and name directory into its own external repository unchanged. It has no runtime dependency on BenchWeave core or the SDK. The SDK is a development dependency.

The starter deliberately implements a made-up, read-only identify and voltage protocol. A pass of its tests demonstrates that the tools work. It says nothing about your instrument.

2. Give the AI the facts and prompts

Open the generated AI-GUIDE.md. It contains five copyable prompts: establish device facts, implement with mocks, demonstrate behaviour, prepare independent review, and prepare release artefacts for owner review.

Provide the exact model, firmware, transport, supported operations and authoritative protocol evidence. Review the AI’s proposed scope before you replace the example. The AI should keep the protocol code separate from the async adapter. The AI should use only the supplied scoped host services. The current gateway loader supports package-relative and standard-library imports. Arbitrary third-party runtime dependencies need more integration work.

For custom hardware, follow Build my own device firmware (main repository) as well. Firmware and the Python plugin have separate builds and compatibility evidence, even when you maintain them in one project.

3. Run the checks

From your plugin project, use the environment that contains the SDK:

pytest
benchweave-sdk check src/benchweave_acme_model100/descriptor.json
uv build

Generated tests cover identify and read, quiet lifecycle, invalid inputs, pre-dispatch cancellation and expiry, and uncertain malformed and transport responses. Extend them for your actual operations, failure modes and evidence. check_operation validates request and result schemas and correlation. check_lifecycle checks a quiet lifecycle and repeated cleanup. Both bound cooperative async calls with a configurable wall-clock timeout. Python that blocks, or code that suppresses cancellation, needs process-level isolation.

Descriptor checks currently cover its schema, matching capabilities and policies, unique parameter names, ordered ranges and transport-provider declarations (next subsection). These are partial checks, not all S01–S18, C01–C12 or M01–M14 obligations. The SDK provides no physical transport provider, complete profile or capture test harness, or hardware qualification certificate.

Standards versions: per-pin validation and the served set

The SDK does not validate against “the” OTDP schema. It vendors a multi-version standards tree. It validates every document against its descriptor’s own pinned version. The otdp_version field decides which bytes answer. A 0.2.0-pinned descriptor validates against 0.2.0’s schemas. A 0.2.2-pinned descriptor validates against 0.2.2’s schemas. Neither is reinterpreted through the active version. Explicit otdp/<version>/<file> lookup keys resolve the same way.

standards-lock.json pins what the SDK carries: one row per retained, in-range (id, version), with yanked-in-interval versions carried but marked. The lock also carries a verbatim mirror of the gateway’s dependency_policy block. The mirror holds the declared ranges, yank records and retirements. Every vendored file is sha256-checked against its lock row at load. benchweave-sdk sync-standards --check re-verifies the committed tree, the stamps and the row-marker and mirror agreement offline, with no gateway checkout.

Pin classification follows that state, under stable refusal prefixes:

  • version_not_served: means that this SDK does not carry the pin. The pin is out of the declared range or was never retained here. The refusal carries the five VR-37 fields: standard, pinned version, supported range, nearest move-to and the migration-note pointer.
  • retired_identifier: means that the pin names a retired version. A retired version was used, is now dead, and will never be reissued.
  • A yanked pin is carried and yank-marked. It stays conforming and validates against its own bytes. A deprecation warning names the derived move-to. The move-to is the highest served version. When nothing is served, the declared range’s lower bound is the guidance. When the pin orders above every served version, the move-to is a downgrade. The warning then carries the exact text (a downgrade — no served version is newer). It does not present the move-to as an upgrade path.

Transport providers (custom transports)

A custom transport may declare a pinned transport-provider contract (OTDP 0.2.1). The declaration is an optional provider object on the transport. It carries the required feature_id and an {id, path, sha256} pin to the reviewed contract document inside your plugin package. benchweave-sdk check verifies the whole declaration offline. It refuses with named, machine-matchable prefixes:

  • provider_feature_missing: means that the provider’s feature_id is not in required_features.
  • provider_transport_undeclared: means that an otdp.transport.* feature is required but no provider on the custom transport declares it. It also fires when the provider’s feature_id is not a sanctioned otdp.transport.<name>/<semver> id. Such a declaration is one that nothing admissible can ever back.
  • unknown_otdp_feature: means an otdp.* feature id that is neither corpus-known nor declared through a provider. The namespace is corpus-owned. A typo or a fake version does not pass.
  • provider_contract_missing: means that the pin does not resolve to a contained regular file. Provider pins are descriptor-relative: the contract ships beside your descriptor, and subdirectories are allowed. Provider pins refuse traversal, absolute paths and symlinked components.
  • provider_contract_hash_mismatch: means that the pinned bytes do not hash to the pinned sha256.
  • provider_contract_invalid: means that the pinned document fails one of these checks: the transport-provider contract schema; grammar subschemas that are not valid Draft 2020-12 schemas; kinds that are not unique strings; a kind that shadows a generic transfer kind; identity fields that disagree, including disagreement with the descriptor’s own declaration.

A provider in the wrong place refuses through the ordinary error surface for schema validation. The wrong places are a non-custom transport and the missing adapter integration mode. Provider-object shape faults refuse through the same surface. A custom transport without a provider stays valid. That is the honestly-incomplete posture, not a refusal. The Python entry points behind these checks are validation.validate_transport_provider (contract documents) and validation.verify_provider_pin (pin resolution).

These checks prove the declaration well-formed and self-consistent only. Only the gateway can prove the grant: that your host admitted the pinned contract and that connection_key resolves to a connection it backs. An offline pass is not an admission claim.

Local UI preview (simulated)

benchweave-sdk check-ui and check-preset validate presentation candidates offline. preview-ui previews a plugin project. It serves the standalone host on the scripted mock transport, on loopback only. The command needs the [server] extra. On a default installation, it refuses with benchweave_sdk_server_extras_missing: and exit 2. preview-ui never leaves the mock transport. Every scenario, observation and control receipt is simulated and labelled as such. The preview path has no gateway, registry or hardware reach. The served document’s shape is pinned by plugin-ui-preview-v1 (main repository). The emitter validates it. The renderer_version field of the served document reports the version of the SDK that emitted it. Previews render the manifest’s declared plots and channel hints from per-scenario snapshot values. Each observed target gets one simulated value. Previews do not show observation history.

For real hardware, use benchweave-sdk-server serve --transport serial directly. The command starts without a device path. Scan from the index page, then bind one listed port with its Bind button. Bind and Unbind need the operator action token. serve prints this token at startup. The launch URL carries it. The command also writes it to operator-action-token.json under the capture root. The binding is stored in device-bindings.json under the working directory. Name another file with --bindings <path> or BENCHWEAVE_STANDALONE_BINDINGS. The stored binding survives a restart: the next serve connects to the bound port without a device path. A connect refuses not_ready with reason binding_stale when the bound port is gone. The device page shows a re-pick action beside this refusal. A bind also refuses when the picked port is no longer present, or when the port now carries a different USB serial than the scan recorded. Scan again, then pick one of the listed ports. A connect also refuses when the unit answers the identify request with a different manufacturer or model than the descriptor declares. The session stays closed. Pass --device <path> for the headless form. The host then serves that constant path. The bindings file is still opened and validated at startup. The stored binding is not consulted to resolve the endpoint in this form. The pick flow in the UI can still write the file.

The SDK’s runtime dependencies are Click, jsonschema (with the referencing, rfc3339-validator and rfc3987 format backends) and Rich. This repository’s pyproject.toml declares them all. In a main-checkout layout, the file is at packages/sdk/pyproject.toml. The wheel vendors the canonical contract JSONs and the pure presentation validator.

Operational notes: preview-ui is an interactive tool. It serves until you interrupt it, by design. It exits 0 on SIGINT as a normal stop. The default port is 8477. Pass --port to change it.

Caution

Under the explicitly acknowledged --allow-network mode, request bodies have no handler timeout. Only untrusted peers are affected. Do not expose the preview to untrusted peers.

The plugin stylesheet override (preview.css)

A plugin can declare a local stylesheet override for its preview pages. The manifest declares an assets[] row whose path is exactly preview.css. The row also declares the file’s sha256.

The manifest row. Add an assets[] row to the manifest. Give path the value preview.css. The value must be exactly preview.css. Give sha256 the digest of the file bytes. Put the file in the plugin’s resource directory.

{
  "assets": [
    { "path": "preview.css", "sha256": "5f2a9c8e1b7d4f6a0c3e9d2b8a7f1e4c6d5b3a9f0e8c7d6b5a4f3e2d1c0b9a87" }
  ]
}

The serve-time checks. At serve time the host checks the file bytes against the row’s sha256. The file can be 64 KiB at most. The host serves it as text/css. The override loads last on your plugin’s preview pages. The gateway never serves a plugin stylesheet override.

The refusal. If the bytes do not match the pin, the host refuses the override. The page renders with the default tokens and carries the marker attribute data-bw-override-refused on the html element. A direct fetch of the file returns 403 with the prefix standalone_preview_override_refused:. A fetch when the manifest declares no override returns 404. The host also refuses an unreadable file or a file above the cap.

The cascade. The override wins at equal CSS specificity, because it loads last. The host rules for the lanes figure sit at specificity (0,2,0). An example selector is .bw-lanes__segment[data-bw-state-kind="high"]. To change such a rule, an override must copy the exact selector. The reduced-motion rules are an accessibility floor. They carry !important, and an override cannot reach them.

The theme. The preview uses the system color scheme, and it renders light and dark. The behavior is the same as the gateway console.

The capture page. A page that carries a digital_lanes plot leads with the lane traces. The reading tiles sit below as the secondary surface. A page without a lanes plot keeps the tile-first layout.

The lanes figure today. The figure shows the declared structure only. The acquisition line shows Acquired 0 samples · plotted 0. A channel hint visible: false hides that lane in the preview. The preview fixtures carry no capture data yet, so the viewport toolbar does not appear today. A later release will add capture data to the preview fixtures, and the toolbar will then appear. The paragraphs below describe these controls.

The viewport toolbar. A lanes figure over a capture with samples carries a viewport toolbar below it. The toolbar holds the controls ◀, Full, Zoom ×½ and ▶. The server computes the window for each control. The ◀ and ▶ controls move the shown window back or forward by half its size. The Zoom ×½ control shows the middle half of the shown window. The Full control shows the whole capture again.

The rendered controls. The ◀, ▶ and Zoom ×½ controls do not render when their action has no effect. The Full control always renders. A capture without samples renders no toolbar at all.

The window swap. Each toolbar control asks the server for a window of the capture. The request carries two parameters, from and to. Both parameters are absolute sample indices, and sample 0 is the first sample of the capture. The server renders the same lanes against the requested window. The browser then replaces the whole viewport block, figure and toolbar, with the served block. The page refreshes its readings on its own, and a refresh never touches the lanes region.

The served window. The server clamps a requested window into the capture. A request before sample 0 clamps to sample 0, and a request beyond the last sample clamps to the last sample. The served block carries the served window in data-bw-window, which names the first and the last shown sample. The axis label names the served sample span, for example samples 500–999. The acquisition line reports the counts of the shown window. The block also carries the marker data-bw-window-clamped when the served window differs from the request.

The viewport refusal. The server refuses a window request that it cannot serve. The refusal returns status 400, and its body starts with viewport_window_refused:. An empty window or an inverted window refuses. This can happen when the capture changed after the server built the toolbar. The browser then shows a notice in the viewport region. The notice tells the operator to push Full, and the Full control always serves.

Mock transports (offline rehearsal)

The standalone host serves a plugin on a scripted mock transport when you run serve --transport mock or preview-ui. For serve, the mock transport is the default. The script comes from the evidence file of the plugin, src/<package>/vectors.json. The file declares which of the two transaction dialects the adapter uses. The host serves the mock that matches the dialect.

The two dialects. Declare the dialect with the file-level transaction_dialect key. If you omit the key, the file is a stream_exchange script. Every row is a {"request": ..., "response": ...} pair of strings. The mock serves whole-dict exchanges. This is the shape of the starter protocol. The request is ID?\n and the response is one line.

If you declare "transaction_dialect": "send_receive", the rows form a binary section 8.1 SEND/RECEIVE conversation. The strings of each row are frames. The stream_send of the adapter must match the next scripted request frame exactly. The mock serves stream_receive with the exact outcomes of the serial backend. The section “A standalone runtime around the writer” below states these outcomes.

Encoding. By default, the bytes of a row are the ASCII bytes of the row string. Set "encoding": "hex" at file level, or on one row, for binary frames. A string is then a hex byte string, for example "response": "53544154000a". The ascii encoding refuses a string that is not ASCII. The hex encoding refuses an odd-length string or a non-hex string. Both refusals give the row.

The row contract. Every row carries a response. A row without request is a response-only row. A response-only row is a device-initiated frame that the mock releases to the inbound stream. The release happens in row order, when the conversation reaches the row. Response-only rows need transaction_dialect: "send_receive". The default dialect cannot contain an unsolicited frame.

A row can also carry a name. Diagnostics refer to a row by its name or by its index.

How selection works. The host reads the declaration at connect time and builds the matching mock host. Nothing else changes. serve --transport serial never reads vectors.json. The script is rehearsal evidence, not device configuration.

How the script loops. The mock detects a repeating unit from the captured rows. The rows before the first full unit are the establishment. The mock serves these rows once. The unit then serves every later cycle in captured order.

A capture needs at least two full units of evidence. A capture with no derivable unit is not an error. Two causes are usual. A truncated capture is one cause. A polled value that varies across occurrences of the same request is another cause. Such a capture then obeys the plain tail rule. Under this rule, row 0 is the establishment. The whole tail becomes the cycle. The cycle repeats in captured order on every restore.

A repeating unit must contain at least one row with a request. A response-only row releases once per cycle pass. A captured burst drains once. Then the line goes quiet. When the adapter consumes a row with a request, the cycle re-arms.

What the mock can and cannot rehearse. The send_receive mock reproduces exactly the RECEIVE outcomes of the serial backend: exact_bytes precedence over termination, the lf and crlf terminators, the overlong-line discard, the quiet-line {"data": b""}, and a partial frame retained across a deadline. The mock releases unsolicited frames strictly in row order. The mock cannot rehearse the interleaving that a real streaming device produces between rows. The mock does not serve can_receive, can_send, i2c_transfer or spi_transfer. A plugin that needs those kinds needs a real transport.

The timings of the mock are not the timings of the device. The quiet line answers at once, not after the quiet window of the backend. A receive that cannot complete raises the TimeoutError of the backend at once. The mock does not wait until the deadline.

For this reason, a rehearsal proves the command flow, not the timing budget. A capture that fits its timeout_ms on the mock proves nothing about the hardware budget. Budget-fit evidence needs --transport serial and the device.

The bound, in simple words. The discipline of the mock is exact. When a frame that the adapter SENDs differs from the frame of the scripted row, the rehearsal fails. The failure gives the row and both frames. The mock never guesses what the device meant.

But a script can prove only what its rows record. A script that does not match the real device proves nothing about the device. Use --transport serial and the hardware for interleaving, for non-serial transports, for timing, and for any claim about the device.

Capture without a gateway (standalone writer)

Plugin and bench development happens on machines that never run the gateway. benchweave_sdk.capture.StandaloneCaptureWriter implements the three capture methods that an adapter calls in gateway mode: artifact_append, artifact_finalise and artifact_abort. The writer works over one directory per capture on a local filesystem. It is three of the eight CaptureServices members. Device I/O still comes from a transport that a composing runtime supplies.

from benchweave_sdk.capture import StandaloneCaptureWriter

writer = StandaloneCaptureWriter()          # or StandaloneCaptureWriter(some_path)
await writer.artifact_append("cap-demo-1", chunk, context)
manifest = await writer.artifact_finalise("cap-demo-1", {
    "format": "waveform_f64le",
    "started_at": "2026-09-22T00:00:00Z",
    "sample_count": 1024, "sample_interval_s": 0.001, "unit": "V",
}, context)

The capture root is an explicit constructor argument. Next comes the BENCHWEAVE_CAPTURE_DIR environment variable. Last comes captures/ under the current working directory. A relative root resolves against the working directory when you construct the writer.

The writer refuses a root inside the installed package tree. A reinstall or upgrade wipes such a root, and the SDK’s inventory verification refuses unlisted files. The writer also refuses a BENCHWEAVE_CAPTURE_DIR that is set but empty or whitespace-only. It also refuses a root that exists but is not a directory.

Caution

A reinstall or an upgrade deletes a capture root inside the installed package tree. Do not put the capture root there.

Each capture event is one directory. The directory holds manifest.json, a staging/ tree while chunks accumulate, the primary artefact, and an optional renderings/ tree that the plugin writes itself. The primary artefact name depends on the format: <capture_id>.f64 for waveform_f64le, .bin for raw_binary, .csv, .txt or .vcd for declared extension formats, and .data otherwise.

The manifest’s sha256, byte_length and artifact_id (art-<sha256>) are computed over the real published bytes. Standalone extras live under x-standalone-* keys. The finalise format must be in the constructor’s declared set. The set comes from formats=, and it holds the two core formats by default. An empty set refuses everything. The writer refuses a capture that appended nothing. It does not publish an empty capture. It also refuses a waveform_f64le capture whose byte length is not sample_count × 8, as the gateway refuses that too. An abort deletes only staging/ and the in-flight primary. A published capture and renderings/ survive an unconditional finally-cleanup abort.

Standalone does not apply the gateway’s two host-side admission checks: the descriptor’s capture_limits (G2) and the host’s storage quota (G3). A capture that finalises standalone can still be refused by a gateway.

There is no automatic import of standalone captures into the gateway. Ingest is a separate, deliberate path.

A standalone runtime around the writer (worked example)

The writer is three of the eight members. A bench script that drives an adapter without a gateway needs the other five as well: clocks, evidence, and the transport. The example below is one complete CaptureServices object for a device on a serial port. A contributor fork of the ADC board wrote a serial host for the same purpose, and this example is adapted from it.

It implements the three scoped stream transactions of OTDP §8.1 exactly. It adds one behaviour that the spec leaves unspecified: the quiet-line answer below. The transactions are stream_send, stream_receive and stream_exchange. In them, exact_bytes takes precedence over termination. Only lf and crlf terminators exist here, because serial has no eom. Unspecified fields are refused. An incomplete frame is never returned as complete.

A receive that sees no bytes at all for a short quiet window answers {"data": b""}. Nothing was offered. Adapters read that as a quiet line, not as an error. The DPS-150 plugin’s telemetry drain ends on exactly that answer. Once a frame starts, it is only ever returned whole. Bytes already read when a deadline expires stay buffered for the next receive, so a slow reply is not lost.

The capture methods delegate to StandaloneCaptureWriter. This is author-side tooling, not an SDK module. Copy it into your project and adapt it.

import asyncio
import json
import time
from datetime import UTC, datetime
from pathlib import Path
from typing import Any

from benchweave_sdk.capture import StandaloneCaptureWriter

_TERMINATORS = {"lf": b"\n", "crlf": b"\r\n"}
_RECEIVE = {"max_bytes", "termination", "exact_bytes"}
_FIELDS = {
    "stream_send": {"kind", "data"},
    "stream_receive": {"kind"} | _RECEIVE,
    "stream_exchange": {"kind", "data"} | _RECEIVE,
}


class BenchContext:
    """A minimal OperationContext for bench use: identity, deadline, cancellation."""

    def __init__(self, operation_id: str, timeout_s: float,
                 dataset_id: str | None = None) -> None:
        self.operation_id = operation_id
        self.dataset_id = dataset_id
        self.deadline_monotonic = time.monotonic() + timeout_s
        self.dispatched = False
        self._cancelled = False

    def cancel(self) -> None:
        self._cancelled = True

    def is_cancelled(self) -> bool:
        return self._cancelled

    async def mark_dispatch_started(self) -> None:
        self.dispatched = True


class SerialStandaloneHost:
    """All eight CaptureServices members over one serial port, without a gateway.

    ``port`` is anything with pyserial's ``write(bytes)``, ``read(size) -> bytes``
    (returning early on the port's own short timeout, which bounds how quickly a
    deadline or cancellation is noticed) and ``close()``. ``quiet_s`` is how long
    a receive waits for its first byte before answering that nothing was offered.
    """

    def __init__(self, port: Any, *, writer: Any = None, evidence_path: Path | None = None,
                 max_frame_bytes: int = 4096, quiet_s: float = 0.1) -> None:
        self._port = port
        self._writer = writer if writer is not None else StandaloneCaptureWriter()
        self._evidence_path = evidence_path
        self._max_frame = max_frame_bytes
        self._quiet_s = quiet_s
        self._pending = bytearray()  # read from the port, not yet returned
        self._closed = False
        self.evidence: list[dict[str, Any]] = []

    # clocks
    def monotonic(self) -> float:
        return time.monotonic()

    def utc_now(self) -> str:
        return datetime.now(UTC).isoformat(timespec="milliseconds").replace("+00:00", "Z")

    # transport (OTDP section 8.1 stream transactions)
    def _live(self, context: Any) -> None:
        if context.is_cancelled() or self.monotonic() >= context.deadline_monotonic:
            raise TimeoutError("operation cancelled or expired")
        if self._closed:
            raise ConnectionError("serial port is closed")

    async def transfer(self, transaction: dict[str, Any], context: Any) -> dict[str, Any]:
        kind = transaction.get("kind")
        if kind not in _FIELDS or set(transaction) != _FIELDS[kind]:
            raise ValueError(f"not a section 8.1 stream transaction: {sorted(transaction)}")
        receive = None
        if kind != "stream_send":
            receive = self._receive_bounds(transaction)
        if kind != "stream_receive" and not isinstance(transaction["data"], bytes):
            raise ValueError("data must be bytes")
        self._live(context)
        if kind != "stream_receive":
            data = transaction["data"]
            try:
                sent = await asyncio.to_thread(self._port.write, data)
            except OSError as error:
                raise ConnectionError(f"serial write failed: {error}") from error
            # pyserial reports a short count without raising when a pending write
            # is cancelled, or under write_timeout=0. None means the port reports
            # no count (pyserial's own rs485.RS485 and cp2110:// ports).
            if sent is not None and sent != len(data):
                raise ConnectionError(f"serial write reported {sent} of {len(data)} bytes")
            if kind == "stream_send":
                return {}
        return {"data": await self._receive(*receive, context)}

    def _receive_bounds(self, t: dict[str, Any]) -> tuple[int, bytes, int]:
        max_bytes, termination, exact = t["max_bytes"], t["termination"], t["exact_bytes"]
        if not isinstance(max_bytes, int) or isinstance(max_bytes, bool) \
                or not 1 <= max_bytes <= self._max_frame:
            raise ValueError(f"max_bytes must be 1..{self._max_frame}")
        if termination not in _TERMINATORS:  # serial does not support "eom"
            raise ValueError("termination must be 'lf' or 'crlf' on a serial port")
        if exact is not None and (not isinstance(exact, int) or isinstance(exact, bool)
                                  or not 0 <= exact <= max_bytes):
            raise ValueError("exact_bytes must be None or 0..max_bytes")
        return max_bytes, _TERMINATORS[termination], exact or 0

    async def _receive(self, max_bytes: int, terminator: bytes, exact: int, context: Any) -> bytes:
        quiet_until = self.monotonic() + self._quiet_s
        while True:
            if exact:  # a positive exact_bytes takes precedence over termination
                if len(self._pending) >= exact:
                    return self._take(exact)
            else:
                end = self._pending[:max_bytes].find(terminator)
                if end >= 0:
                    return self._take(end + len(terminator))
                if len(self._pending) >= max_bytes:
                    self._take(max_bytes)  # discard, so the next call can resynchronise
                    raise ValueError(f"no terminator within {max_bytes} bytes")
            if not self._pending and self.monotonic() >= quiet_until:
                return b""  # nothing offered: a quiet line, not an error
            self._live(context)  # a partial frame stays pending across the timeout
            want = (exact or max_bytes) - len(self._pending)
            try:
                chunk = await asyncio.to_thread(self._port.read, max(1, want))
            except OSError as error:
                raise ConnectionError(f"serial read failed: {error}") from error
            self._pending += chunk

    def _take(self, count: int) -> bytes:
        out = bytes(self._pending[:count])
        del self._pending[:count]
        return out

    async def close_transport(self, context: Any) -> None:
        if not self._closed:
            self._closed = True
            await asyncio.to_thread(self._port.close)

    # evidence
    async def record_evidence(self, entry: dict[str, Any], context: Any) -> None:
        record = {**entry, "at": self.utc_now(), "operation_id": context.operation_id}
        self.evidence.append(record)
        if self._evidence_path is not None:
            with self._evidence_path.open("a", encoding="utf-8") as stream:
                stream.write(json.dumps(record, sort_keys=True) + "\n")

    # capture: the SDK's standalone writer
    async def artifact_append(self, capture_id: str, data: bytes, context: Any) -> None:
        await self._writer.artifact_append(capture_id, data, context)

    async def artifact_finalise(self, capture_id: str, metadata: dict[str, Any],
                                context: Any) -> dict[str, Any]:
        return await self._writer.artifact_finalise(capture_id, metadata, context)

    async def artifact_abort(self, capture_id: str) -> None:
        await self._writer.artifact_abort(capture_id)

Using it with pyserial and a plugin’s adapter:

import serial  # pyserial, installed separately

port = serial.Serial("/dev/ttyUSB0", 115200, timeout=0.05)
host = SerialStandaloneHost(port, writer=StandaloneCaptureWriter(formats=("raw_binary",)))
plugin = create_plugin()
open_context = BenchContext("open", timeout_s=2.0)
await plugin.open(descriptor, host, open_context)
op_context = BenchContext("bench-1", timeout_s=2.0)
result = await plugin.execute({"operation_id": "bench-1", "verb": "read",
                               "arguments": {"parameter": "voltage"}}, op_context)

Give each operation its own BenchContext. The adapter checks operation_id against the request. tests/test_guide_serial_host.py executes the first block above and drives a freshly scaffolded plugin through it, so the example cannot drift from the SDK or from the generated adapter.

Analyse captures (standalone host)

The standalone host serves the Analyse page at GET /analyse. The page works on the published waveform captures under the host’s capture root. You select captures, set a time window, and read the statistics that the host computes under definition benchweave-analysis/1. Every statistics block states its denominators: the sample count, the null count and the window.

The Captures list offers one check box per capture, and it lists only captures with state published and format waveform_f64le. No other capture can enter an analysis.

Select one or more check boxes, then select Analyse. The page shows one plot and one statistics table per capture. The statistics recompute each time you submit the form. The table rows are count, null_count, min, mean, max, rms and peak-to-peak. A line under each table names the capture’s manifest sha256.

Set the window with the two fields Window start (s) and Window end (s). Both times are seconds from the capture’s own start. The window is inclusive. An empty field leaves that end unbounded.

Both values must be finite numbers, and the start must not exceed the end. A window that breaks these rules refuses with the standalone_report_window_invalid: prefix. The page shows the refusal in a Refused notice.

Drag is the other way to set the window. Press the mouse on a plot and drag across a region. The drag zooms the plot to the region, fills both window fields, and submits the form. The window is then the dragged region. Both fields receive values rounded to 6 decimal places. The window fields alone are the complete path without script.

A sample counts as null exactly when it is NaN. The host counts null samples and excludes them, and it never reads a null as zero. When the window holds no non-null sample, every statistic shows the absence mark —.

The marker editor works on one capture at a time. Select exactly one capture to see it. Each marker row has three fields: a label, a time and a note.

A label is one uppercase letter A to Z. One marker exists per label, and a save with a repeated label replaces the earlier row. The note is free text. The editor pads empty rows to three, and it shows at most 26 rows.

The time is seconds from the capture’s own start. It must lie inside the capture’s displayable span, from 0 to (sample_count - 1) × sample_interval_s. A marker outside that span refuses with the standalone_report_marker_invalid: prefix.

Click a plot to write a time into a marker row. The click fills the row you last touched, or the first row with an empty time. A drag never places a marker. The drag brush owns drags. The marker time fields alone are the complete path without script.

Select Save markers to store the rows. The save rewrites the capture’s analysis.json overlay, format standalone-analysis/1. The save replaces the whole overlay with the editor’s rows, stored sorted by label. Empty rows store no marker. Your unsaved rows also ride the export. When you export, the editor’s current rows become the report’s markers for that capture.

Power modes pair the set’s voltage captures and current captures into rails. The pairing uses the units only, never the capture names. A capture with unit A, mA, uA or nA is a current side. A capture with unit V, mV, uV or kV is a voltage side. Letter case does not matter, and µ reads as u.

Without explicit rails, the first voltage side and the first current side of your selection form rail 1. The next pair forms rail 2, and so on. A current side without a voltage partner keeps its own rail, with rows for the current side only. A voltage side without a current partner forms nothing.

A rail pairs whole captures only. Both sides must declare the same sample interval and the same sample count. A mismatch refuses with the standalone_report_power_pairing: prefix.

The Power mode field names one presentation over the rails:

Mode Works on More input
battery rail 1 Battery capacity (Ah), for the runtime row
dc-dc rails 1 and 2 none
sleep rail 1 current Sleep threshold, in the rail’s current unit
load-step rail 1 none

The battery runtime row appears only when you give a capacity. The runtime is the capacity divided by the mean current. A zero or non-finite mean current omits the row, and no default capacity exists. The sleep block reads not_evaluated with reason no threshold when you give no threshold. An empty class reports its count and shows the absence mark for its mean.

The dc-dc efficiency row stays absent when the host cannot compute it, and a reason names why. The same holds for the load-step resistance row.

Every power label derives from the rail’s actual units. The power label is W only when both sides use the base units V and A. A scaled rail prints its unit product instead, for example V·mA. The charge label is Ah only for unit A, and the energy label is Wh only for a W rail. The host never rescales a value to match a label.

The power section can state an unavailability instead of rows. power_unavailable: no current series means the selection holds no current side. The section then shows no power rows, never a 0 W row. A rail with no voltage side states power_unavailable: no voltage series where the paired rows would be. Its charge row stays, because the charge integrates the current capture alone.

Export and read the analysis report (standalone host)

Select Export report to render one self-contained HTML document. The host writes the pair reports/<report id>.html and reports/<report id>.json under the capture root. The report id is rep- plus the first 16 hex digits of the document’s SHA-256. The id names the document’s own bytes. Identical inputs re-export to the same report. The second export writes nothing new, and the API result answers created: false for it.

The document holds these sections:

  • a banner line STANDALONE — no gateway, plus SIMULATED (a source capture records mock transport) when a source records mock transport;
  • the definitions line, which names the definition ids and states Processed values are host-computed (uncertainty: unknown);
  • the context chart, with every capture’s full extent, the window shaded, and the in-extent markers drawn;
  • the zoom chart, with the window at full detail, only when the window is a proper part of the extent;
  • one window statistics block and one edge timing block per capture;
  • the power section, when a power mode was selected;
  • the assertions table, when the export carried assertion rows from the REST or MCP surface;
  • the markers table, with every stored row and an in-window yes or no;
  • one sources block per capture, with the unit, the sample count, the interval, the byte length, the start time, the full manifest sha256, the device, the firmware, the plugin, the descriptor sha256, the operator, the project and the notes;
  • the footer, with the SDK version, the renderer pin and the re-derivable statement.

Each edge timing block uses definition benchweave-edge/1. The baseline is the mean of the first 15 % of the window’s non-null samples. The final value is the mean of the last 15 %. Each side counts at least 1 sample. A window with fewer than 3 samples, or a flat step, reads detected: no.

The rise_time row is the span from the 10 % crossing to the 90 % crossing. The settle_time row measures from the 10 % crossing. It shows the absence mark when the window’s last sample is still outside the settle band. The settle band is 2.0 % of the step, and the web export always uses this value.

An assertion row uses definition benchweave-assert/1. It names a capture and a lower bound, an upper bound, or both. The verdict is pass, fail or not_evaluated. An empty window is not_evaluated, never pass. A fail row names its reasons, below_min and above_max.

Every charge and energy row carries two coverage rows. One states the integrated span in seconds. The other states how many segments were dropped. A dropped segment is a segment where one sample of the pair is null.

The integral never steps across a gap. When no segment was counted, the value is the absence mark. No data integrated is an absence, not zero energy.

The report uses these marks where a value does not exist:

Mark Meaning
— The window held no data for this value. No value was computed.
n/a (non-finite) Data was present, but the statistic has no finite value.
not_evaluated An assertion found no samples in its window, or a sleep block had no threshold. It never means pass.

Before any number renders, the export reads each primary file again and recomputes its SHA-256. A capture that changed after publication refuses with the standalone_report_primary_mismatch: prefix. A refused export writes nothing, and it never creates the reports directory.

The document holds no clock value, so the same inputs give the same bytes. The JSON file beside the document records the export time, the parameters and the digests. The parameters are a replayable export input. Feed them back to reproduce the document with the same bytes.

Two limits protect the report’s shape. The report draws at most two distinct units. A larger set refuses with the standalone_report_units_unsupported: prefix. Split the export in that case.

A window that holds more than 2000000 samples for a plot refuses with the standalone_report_window_too_large: prefix. Narrow the window in that case. One export names at most 16 captures.

The export pins every source capture. A pinned capture never expires under retention, and it refuses deletion until you unpin it. Without a pin, mcp captures expire after 30 days. The ui and rest sources stay until you delete them. The capture retention section above explains the schedule. After the export, the browser lands on the report at GET /reports/<report id>. ### The MCP stdio session (the shim)

Agents speak MCP over stdio to the standalone host:

benchweave-sdk-server mcp /path/to/plugin-project

The command does one of three things:

  1. Proxy. A started host owns the supervision family beside the bindings document, which comes from --bindings <path>, or BENCHWEAVE_STANDALONE_BINDINGS, then device-bindings.json under the working directory. The start command applies the same resolution. The stdio session then proxies to that host. The browser page and the agent share one host and one device session. The proxy sets no call timeout. A hung host holds the session, and the command says so once on stderr at startup.
  2. Fallback. No host runs, because no pidfile exists or the pidfile names a dead process. The command serves the plugin in-process and prints one line on stderr that says so. No browser can attach to that session. The transport is the scripted mock. Captures in that session are ephemeral and go to a private temporary capture root that the command discards when the session ends normally. A fallback never touches the ambient capture root.
  3. Refuse. A host runs, but the command cannot verify it or use its credential file, or another process holds the library lock of the capture root. The command exits non-zero and names the fix. The status command shows the family’s verdict. When the command cannot verify a family, no lifecycle verb clears it. The refusal names the path of the stale file that remains: the pidfile, or the tokens file when no pidfile exists. Remove that file to resolve the state. The command never serves a second host beside a live one.

A .mcp.json entry:

{
  "mcpServers": {
    "benchweave-sdk-server": {
      "command": "benchweave-sdk-server",
      "args": ["mcp", "/path/to/plugin-project"]
    }
  }
}

The stdio session holds no state of its own: it writes no pidfile, no journal row, no tokens file and no capture-root file. In proxy mode the served tools are exactly the host’s tools. The session answers initialize with instructions that name the mode. The proxy sentence carries the host URL and the pid. The fallback sentence says that no other surface can attach and that the session’s captures are ephemeral. Point the entry at the same bindings document as start, so the supervision family has one anchor.

4. Review compatibility and deployment

Use the AI device integration reviewer (main repository) with the exact revision and test evidence. Its SDK maintenance checklist covers public interfaces, contracts, generated projects, mocks, gateway compatibility and release artefacts.

The gateway now has an explicit load_otdp_plugin and OTDPBridge path for async adapter API 1.1. The bridge mediates the OTDP verbs identify, read, write, capture, stream_subscribe, stream_unsubscribe and invoke: scalar read, scalar write, single-channel capture, streaming subscriptions and pinned-contract invoke. The capture verb comes with pre-dispatch bounds checks, staged appends and a host-computed manifest. The step’s timeout_ms is the capture budget. The release smoke exercises a generated multi-module plugin through identify and read. Separate bridge tests cover write conversion and rejection paths. The OTDP loader path also serves dataset publishing and payload services. The bridge does not mediate the self_test, get_errors and reset verbs. Profile scheduling needs a native asynchronous host. The bridge does not supply it. The gateway retains the existing simulator loader for its separate synchronous interface.

A gateway integrator must first do four things:

  • complete admission;
  • supply the verified manifest and cache, and the descriptor;
  • inject authorised scoped services;
  • use the same monotonic timebase as the host’s deadlines.

The loader rechecks all listed file hashes and imports verified package code and resources in memory. This is an integrity check and namespace isolation, not an operating-system sandbox. Adapters remain trusted Python.

An SDK install does not add a plugin to a running workbench. There is no live device-install endpoint, automatic dependency installer or Docker hot-install command. A Docker deployment still needs persistent storage for verified packages, bench configuration, commissioned host services and activation at an approved idle boundary. See package formats and gateway installation (main repository).

5. Prepare the release, then approve publication

A Python wheel or source distribution and a registry release are different artefacts. Prepare the registry-required identity, exact compatibility, payload inventory, hashes, dependency locks, licence, source and build provenance, SBOM (software bill of materials) and evidence status. benchweave-sdk inventory <prepared-bundle-directory> prints file paths, byte counts and SHA-256 values. It does not create or approve a complete registry manifest.

Before you publish, show the owner the actual built files, the evidence and the remaining gaps. Hardware qualification and local commissioning remain separate. The SDK retains the project’s proprietary licence declaration. Release owners must establish distribution rights before they share it publicly.

After your submission lands as a pull request (PR), track it without asking anyone. benchweave-sdk registry status --publisher <you> --registry-clone <clone-of-benchweave-registry> reports your exact submission set. It also reports every release’s lifecycle timeline, from the committed records alone. The timeline covers publishes, advisories, yanks and transfers. This command is the contributor-facing half of the queue that maintainers see. Stages carry stage_partial flags, because PR-side review activity is not derivable from records. The maintainer’s registry queue reads it when a live gh is available.

Maintenance and release cycle

The main repository’s Package workflow builds the gateway and SDK from source distributions, then tests installed wheels in temporary environments on Linux and macOS. It generates, builds and installs an external plugin outside the checkout. It runs the plugin’s tests. It checks packaged contract bytes and versions. It exercises gateway loading. It rejects tampered code.

Run the same check locally from a checkout of the main BenchWeave repository. The script scripts/sdk_smoke.py lives there, not in this repository. Run:

uv run --no-project --python 3.13 scripts/sdk_smoke.py --out-dir dist/packages

On a published GitHub release, a separate job repeats these checks. It then attaches the SDK wheel and source distribution to that release. A release tag triggers the publish workflow. The workflow builds the source distribution and wheel. It smoke-installs the wheel on five OS and arch lanes (Linux x86-64 and arm64, macOS arm64 and Intel, and Windows). It verifies each installed wheel’s vendored standards with benchweave-sdk sync-standards --check. It publishes to PyPI via Trusted Publishing. No release is published simply by an edit to these files or by a run of the local smoke check.

An installed SDK can verify its own vendored standards offline: benchweave-sdk sync-standards --check compares the packaged lock with the packaged tree and needs no repository checkout. An import of a standards bundle rewrites the source tree, so the command refuses to run outside a checkout (sync_requires_repo_checkout:).

When you change the SDK API, canonical contracts, template, gateway bridge or build tooling, update the affected tests and documentation together. Keep SDK distribution metadata, __version__, generated development dependency and this compatibility statement aligned. The smoke checks guard the installed distribution and contract resources.