setup-enformer_deepmind-remote-tool

Set up the official DeepMind Enformer as a remote tool

Validation status (2026-10-14): the official TF-Hub SavedModel deepmind/enformer/1 ran on the NVIDIA GB10 host (aarch64, 20 cores, 121.6 GiB unified memory, cc 12.1, driver 580.173.02) on the CPU, under kit mode off. This provider's lock has no deviation at all, a first on this standard: 41 of the kit's 53 pins are installed at exactly their pinned version, and the other 12 are nvidia-*-cu12 CUDA runtime wheels that a CPU TensorFlow loads none of. All three weight files match the kit's stock/PINS.json by size and sha256. [gpu] policy = "none" is a measurement, not a preference: tf.test.is_built_with_cuda() is False and tf.config.list_physical_devices('GPU') is empty, because TensorFlow publishes no GPU-capable aarch64 wheel at any version. One 393,216-base window costs ~200 s, and residency buys almost nothing — two consecutive predictions of the same window measured 194.1 s then 205.8 s, so there is no graph tracing to amortise. provider.toml's header has the numbers and says what residency is actually for here. Cross-implementation agreement with the enformer port at 8011 has not been measured. Public publication, cross-user isolation, concurrency and recovery are untested; keep this deployment private.

What this provider is for

Enformer predicts thousands of genomic tracks — CAGE, DNase, ATAC, ChIP — from DNA sequence alone, integrating regulatory signal over ~100 kb. This provider serves the authors' own artifact: the TF-Hub SavedModel they exported, pinned by digest.

One sibling overlaps and the distinction is the whole reason both exist:

• enformer (port 8011) is enformer-pytorch, a community port whose weights were converted from the authors' release, running on the GPU with a 4.12 GiB budget. It is faster and it is what you want for throughput.
• enformer_deepmind (this one, port 8060) is the official TensorFlow SavedModel on the CPU. A converted checkpoint under another framework's kernels does not agree to the last float, so published Enformer numbers reproduce against this one.

They are complements, not alternatives: this one reserves no VRAM, so both can be resident at once.

Prerequisites

• Run from the ToolUniverse repository root on Linux with Python 3.12.3.
• No GPU is required, and none can be used. [gpu] policy = "none" makes the preflight skip every GPU check, which is honest here rather than a shortcut: there is no CUDA code in an aarch64 TensorFlow wheel to reach a card with.
• About 1.0 GiB of disk for the SavedModel and ~600 MB for the environment.
• Python 3.11 for the model environment (the kit's python pin).

Run the standard-library contract check before downloading large dependencies:

python scripts/remote_validation/setup_skill_preflight.py --implementation enformer_deepmind

After exporting provider resources, add --check-provider-env. After the server starts, add --live to verify the exact MCP tool set without running the model.

Create the environment

uv venv --seed --managed-python --python 3.11 ~/enformer-dm-venv
~/enformer-dm-venv/bin/pip install --no-deps -r \
  src/tooluniverse/remote/enformer_deepmind/environment/requirements.lock
export TOOLUNIVERSE_ENFORMER_DEEPMIND_PYTHON=~/enformer-dm-venv/bin/python

Then the kit, which supplies the mode machinery even though the mode is off:

tu remote setup enformer_deepmind --only kit

One thing the index cannot tell you

This stack is the reason to check what a wheel contains, not that it exists. At tensorflow==2.15.1 the aarch64 wheel on PyPI carries no code: a RECORD of six dist-info lines, and

Requires-Dist: tensorflow-cpu-aws ==2.15.1 ; platform_system == "Linux" and (platform_machine == "aarch64")

hands the real library to a different distribution of the same version. Install that with --no-deps and pip list shows tensorflow 2.15.1 while import tensorflow raises ModuleNotFoundError. At 2.17.1, pinned here, the aarch64 wheel is the real library (13,509 RECORD lines) and installs as pinned — which is why this provider needs no deviation and its TensorFlow 2.15 sibling (borzoi) does.

Fetch and verify the weights

tu remote setup enformer_deepmind --only weights

That fills tensorflow-hub's own cache layout under TFHUB_CACHE_DIR:

$TFHUB_CACHE_DIR/c444fdff3e183daf686869692c26e00391f6773c/
    saved_model.pb                              2,310,075 B
    variables/variables.data-00000-of-00001 1,005,172,750 B
    variables/variables.index                       5,893 B

All three are declared in provider.toml with a size and a sha256 and verified before the worker starts. The worker refuses to start when TFHUB_CACHE_DIR is unset rather than silently reaching the network — a provider that fetches at request time is a provider whose weights nobody pinned.

Start the server and call it

export TFHUB_CACHE_DIR=~/enformer-dm-weights
python -m tooluniverse.remote.enformer_deepmind.enformer_deepmind_tool

tu doctor --forward http://127.0.0.1:8060/mcp --json

A request is a DNA string. It is centre-cropped or N-padded to 393,216 bases, of which the network reads the central 196,608 — so the cost does not depend on the length you send: an 8 kb sequence costs the same ~200 s as a 393 kb one. Send the window you mean.

The reply carries centre (the centre-bin value across every track of the chosen head: 5,313 human, 1,643 mouse) and profiles (the full 896-bin profile of any track named). centre is deliberately the same key the port at 8011 answers with, so the two implementations can be compared field for field.

Bounds, and what happens at them

bound	value	at the bound
sequence length	786,432 (2x the input length)	refused with the ceiling named; anything past 393,216 is centre-cropped anyway
letters	A, C, G, T, N	anything else refused by name; N becomes an all-zero column, as upstream's notebook feeds it
organism	human or mouse	refused with both named
tracks	at most 32 profiles	refused with the count; each profile is 896 floats travelling as JSON
concurrency	one request at a time	a second request queues on the worker's lock

Every bound is checked in the tool and in the worker. A ceiling only the worker knows is a ceiling a caller discovers by having a request fail after three minutes of work.

Why the kit is off

opt/enformer_deepmind_opt/ops/edm_ops.so has ELF e_machine 0x3e — x86-64 — and the repository ships no aarch64 build of it, as for all eight kits that carry prebuilt host objects. The levers behind it are CUDA kernels, which a CPU-only TensorFlow could not reach even with an aarch64 build. The kit says so itself without being asked:

$ PYTHON=~/enformer-dm-venv/bin/python bash run.sh check
[enformer-deepmind-opt] NOT ACTIVE mode=exact reason=no CUDA device visible (tf.config.list_physical_devices('GPU') is empty)

exit 3. On x86-64 with a CUDA TensorFlow, set [kit] mode = "exact" and its fourteen levers engage; tu remote setup refuses the mismatch by name before installing.

What the Platform evidence covers, and what it does not

This implementation was added after the authenticated 2026-08-16 Platform matrix and was not part of it. That matrix imported each implementation it covered as unpublished owner drafts and invoked them through /expert-sessions/{id}/test; this provider was imported as no such draft and invoked through no such route. It has not been exercised through the Platform at all: the evidence above is local, on one card, through the loopback endpoint. Public publication, independent-caller authorization/isolation, saturation, and persistent supervision are untested.

Authorize once, then share with one short command

Run once per machine, and again after key rotation:

tu remote login

Then each private share is:

tu remote share enformer_deepmind

Override the defaults only when needed:

tu remote share enformer_deepmind --name my-enformer-deepmind-remote --workers 1

Sharing automatically starts the browser flow when the key is missing, expired, or revoked. A malformed or revoked explicit TOOLUNIVERSE_SERVICE_KEY fails fast instead of being silently replaced. Use tu remote logout to remove only the locally stored key.

Check discovery and relay privately

tu doctor --forward http://127.0.0.1:8060/mcp --json
tu serve --share --forward http://127.0.0.1:8060/mcp --name validation-enformer-deepmind --workers 1

Run a verified example

The Streamable HTTP endpoint is http://127.0.0.1:8060/mcp. In a second activated shell run:

Operation: run_enformer_deepmind_predict

{"sequence":"ACGTACGT","organism":"human","tracks":[0,4100]}

That eight-base request is a plumbing check, not a result: anything shorter than 393,216 bases is N-padded, so the model sees almost nothing but zero columns and the numbers it returns mean nothing. It costs the same three minutes as a real window. The sequence the acceptance runs used -- and the one whose sum(centre) of 4606.426440 reproduced across two requests -- is src/tooluniverse/remote/enformer_deepmind/smoke/input.txt, which is too long to inline; the block below reads it when it is there and falls back to the short request when it is not.

Invoke the example through the live local MCP endpoint:

python - <<'PY'
import asyncio
import json
import pathlib
from fastmcp import Client

async def main():
    arguments = json.loads('''{"sequence":"ACGTACGT","organism":"human","tracks":[0,4100]}''')
    smoke = pathlib.Path(
        "src/tooluniverse/remote/enformer_deepmind/smoke/input.txt"
    )
    if smoke.is_file():
        arguments["sequence"] = smoke.read_text().strip()
    async with Client("http://127.0.0.1:8060/mcp") as client:
        result = await client.call_tool("run_enformer_deepmind_predict", arguments)
        print(result)

asyncio.run(main())
PY

Expected success shape: implementation, organism, n_bins 896, n_tracks 5313, a bounded centre array and timing. Check scientific meaning, finite values, output bounds, invalid-input behaviour, and the absence of paths, secrets and traces.

Troubleshooting

• import tensorflow fails after a clean install. You installed tensorflow==2.15.1 or another version whose aarch64 wheel is a shim. See "One thing the index cannot tell you" above; this provider pins 2.17.1, whose aarch64 wheel is the real library.
• The worker exits 3 naming TFHUB_CACHE_DIR. It is unset. Set it to the directory --only weights filled; the worker never fetches.
• hub.load tries to reach the network. The pinned cache key c444fdff3e183daf686869692c26e00391f6773c is missing from $TFHUB_CACHE_DIR. Re-run --only weights.
• A request takes three minutes and you expected seconds. That is this provider: CPU-only, and the second call is not faster than the first. Use run_enformer_predict at 8011 for throughput.