qwen-mm-plugins-mhs

Qwen-MM-Plugins MHS

You have qwen-mm-plugins-mhs MCP tools available. They operate real hardware. MHS is to hardware what MCP is to tools: one fixed six-tool surface, with everything device-specific behind an adapter that the hardware's owner runs. You never learn a per-device API — you ask the device what it can do.

The tool surface does not grow when new hardware appears. A new device shows up in mhs_discover because someone started an adapter for it.

Check the qwen-mm-plugins-mhs tools in your tool list for full schemas. For worked examples, see the MHS cookbook.

The loop

1. mhs_discover — always first. Device ids come from here, never from a guess. Ids look like <adapter>/<device_id>; a bare id works when only one adapter has it.
2. mhs_meta_info — before your first mhs_write to a device. This is where you learn the capability's parameters, units, permitted ranges, and which writes need confirmation. Skipping it means guessing at values that move physical things.
3. mhs_read — sensor values, current settings, camera frames. A frame comes back as an image you can look at directly. Read-only; safe to repeat.
4. mhs_write — the only tool that changes the world. See below.
5. mhs_health_check — after anything unexpected, and before a sequence of commands. Never cached. Call it with no device_id to survey everything when you don't yet know what's wrong.
6. mhs_reset — mode="soft" clears an error and returns the device to idle; mode="estop" stops it now.

Writing to hardware

• Read the metadata first. Ranges and units are declared per device; nothing is universal.
• A refusal is information, not an obstacle. If a write is refused for exceeding a hard safety limit, the device has said that value is unsafe. Do not look for a way around it — report it. confirm=true cannot override a hard limit and trying is not a plan.
• confirm=true is for deliberate acts, not for retrying. Use it when metadata says a capability requires confirmation, or to exceed a soft limit you have a specific reason to exceed. If a human hasn't asked for the consequence, don't confirm it.
• A failed write is not automatically retryable. Call mhs_health_check first. Repeating a command at a device in an error state is how a stuck actuator becomes a broken one.
• Say what you actually did. Report the device id, capability, and values you sent, and what the device answered — not just "done".

When something is wrong

If a device behaves unexpectedly, or you are unsure whether a command took effect, mhs_reset with mode="estop" first and diagnose second. It needs no confirmation, because anything standing between you and a stop is a hazard. Then tell the user what you stopped and why.

If a device does not implement reset, the tool says so — that device cannot be stopped through you, and the user needs to know that immediately.

Configuration

Adapters are listed in ~/.qwen-mm-plugins/mhs-devices.json (override with QWEN_MM_MHS_DEVICES):

{"adapters": [{"name": "lab-cam", "url": "http://192.168.1.20:8800"}]}

If that file is missing, the tools say so and show the shape to create. mhs_discover reloads it and queries every adapter on each call; device lists and health are never cached. An empty adapters list is valid. QWEN_MM_MHS_CACHE_TTL (default 60s) applies only to metadata, which discovery and reset also invalidate.

No hardware to hand? Run the bundled mock adapter and point the registry at it:

python3 -m pip install "msgpack>=1.1,<2"
python3 references/mock_adapter.py --port 8800

It serves a simulated camera (with a hard limit on exposure and a soft limit on gain) and a lamp whose power write requires confirmation — enough to exercise the whole loop.

Adding hardware

Nothing about a new device belongs in this plugin. An adapter is a plain HTTP server — any language — and it is registered by one line in the registry file. The full contract is references/adapter_protocol.md, and references/mock_adapter.py contains working device implementations. Its shared HTTP + MessagePack layer is references/adapter_server.py.

You can write the adapter yourself

If the user has hardware you can reach — a device file, a /sys node, a serial port, a gRPC or HTTP service, a vendor CLI — but no MHS adapter for it, write one and use it in the same session. Nothing needs restarting: the registry file is re-read on every call, and devices are discovered from adapters at runtime.

1. Read references/adapter_protocol.md. Do not guess the shapes from these tool descriptions.
2. Copy references/mock_adapter.py together with references/adapter_server.py and replace the device's read/write/health/optional reset methods with real I/O. Keep HTTP routing in the shared server; keep parameter validation, resource lifecycle, and cancellation in the device. Keep it outside this plugin's directory — an adapter belongs with its hardware.
3. Install msgpack>=1.1,<2 in the adapter environment. Return image data as raw bytes. Run it in the background on a free port, and check it started before going further.
4. Check it against the protocol before registering it:

   python3 -m qwen_mm_plugins_mhs.verify http://127.0.0.1:8800
   

   This is read-only — it never writes and never resets — and it names exactly what is wrong (missing blocks, malformed image data, a forgotten direction, an unbounded writable parameter, an error path that answers 200). Fix everything it reports as ✗ before going on; read the ! lines too. Do not skip this because the adapter "looks right" — the failures it catches are the ones that otherwise surface later as a tool error you cannot explain.
5. Add it to ~/.qwen-mm-plugins/mhs-devices.json (create the file if absent, and preserve any adapters already listed — do not overwrite someone else's registry). Write the updated JSON to a temporary file in the same directory, then atomically replace the registry so a concurrent call cannot read a half-written file.
6. mhs_discover — the device is there immediately. The same call reflects removed registrations, changed adapter addresses, and unreachable adapters. Then proceed with the normal loop.

Rules for an adapter you wrote yourself, because you are now on both sides of the safety boundary:

• Start read-only. Declare direction: "read" for every capability until the user has confirmed they want the device actuated. A read-only declaration is enforced by the host, so it costs nothing and it means an accidental mhs_write cannot reach the hardware at all.
• Declare real safety_limits. The host enforces exactly what you declare, so this is the cheapest place to make an unsafe command impossible. Do not declare a range wider than you can justify from the device's documentation.
• Mark consequential writes requires_confirm: true.
• Never fake success. Writes and resets must return a boolean ok: true only when accepted, false when declined. Return 405 when reset is unsupported. A missing or invalid ok leaves execution unconfirmed, including whether an emergency stop completed. You will be the one misled later.
• Tell the user what you built before you use it to move anything — what device it talks to, which capabilities it exposes, and which of them can change physical state. An adapter you wrote has had no review; say so.

Relationship to other capabilities

• A camera frame from mhs_read is an ordinary image: analyze it with qwen-mm-plugins-api (vision_chat, ocr, grounding) or annotate it with qwen-mm-plugins-core (crop, draw_bbox).
• On a text-only host (QWEN_MM_NATIVE_MODE=0) frames arrive as generated captions instead of images; the loop is unchanged.
• This capability reads no files and calls no cloud API — it only talks to the configured adapters.