SuperAPI AI Directory
Partner中文AI newsFavorites
Skillssetup-borzoi_calico-remote-tool

setup-borzoi_calico-remote-tool

Agents & models

View repository

Updated 10-20 05:55

About this skill

Set up calico's Borzoi variant scoring as a remote tool

Validation status (2026-10-14): calico's own borzoi_sad.py at 5c93582 on baskerville at 544073b, fold 0, reverse-complement ensemble, ran on the NVIDIA GB10 host (aarch64, 20 cores, 121.6 GiB unified, cc 12.1, driver 580.173.02) on the CPU, under kit mode off. The worker holds the model but reimplements nothing numeric: it imports stock's own initialize_output_h5, write_snp_len and write_pct out of borzoi_sad and calls baskerville's own snp_seq1, untransform_preds and seqnn model in stock's order, then writes the same sad.h5. This kit's exact mode is the first on this standard that actually runs on this host — it ships no prebuilt host objects and its levers are Python data-path rewrites, not CUDA kernels — and it is byte-identical to stock across three pairs, all 16 datasets; it is still set to off because no speed advantage could be established here (within-mode spread 33 s against a between-mode difference of 14.6 s, on a host whose load climbed through the measurement). provider.toml's [kit] section carries every number. [gpu] policy = "none" is a measurement: TensorFlow publishes no GPU-capable aarch64 wheel at any version. About 255 s per variant; four variants maximum. Public publication, cross-user isolation, concurrency, recovery and indels are untested; keep this deployment private.

What this provider is for, and what it is not

Regulatory variant scoring the way the Borzoi paper reports it: each variant's reference and alternate 524,288-base windows through the fold-0 network under a reverse-complement ensemble, untransformed with upstream's targets table, strand-summed from 7,611 tracks down to 6,385, scored as SAD, logSAD, D2, logD2.

It is not the same operation as run_borzoi_variant_effect at port 8012, which is borzoi-pytorch — a community port — taking two sequences and returning a central-bin delta on the GPU in seconds. Reach for that one for throughput, and for this one when the paper's statistics are what you need.

Prerequisites

  • Run from the ToolUniverse repository root on Linux with Python 3.12.3.
  • No GPU is required and none can be used on aarch64.
  • Python 3.10 for the model environment (the kit's python pin).
  • About 3.9 GiB of weights and genome, plus ~1 GiB of environment, in a directory that must be writable — see below.
  • ~10 GiB of free RAM per request. Peak RSS measured 10,011,120 KB.

Run the standard-library contract check before downloading large dependencies:

python scripts/remote_validation/setup_skill_preflight.py --implementation borzoi_calico

After exporting provider resources, add --check-provider-env. After the server starts, add --live to verify the exact MCP tool set without running the model.

The weights directory must be writable

Unlike every other provider on this standard, a read-only weights mount does not serve this one: pysam.Fastafile builds hg38.ml.fa.fai beside the genome the first time it opens it (borzoi_sad.py:246). launch.py checks os.access(root, os.W_OK) and refuses by name, rather than letting the worker die inside pysam with an errno nobody will connect to the mount.

Create the environment

uv venv --seed --managed-python --python 3.10 ~/borzoi-kit-venv
~/borzoi-kit-venv/bin/pip install --no-deps -r \
  src/tooluniverse/remote/borzoi_calico/environment/requirements.lock
export TOOLUNIVERSE_BORZOI_CALICO_PYTHON=~/borzoi-kit-venv/bin/python

Then the two upstream checkouts, which the kit ships as archives so nothing is cloned at setup time (stock/borzoi-5c93582.tar.gz, stock/baskerville-544073b.tar.gz; an archive install has no .git, so setuptools_scm needs the version handed to it):

SETUPTOOLS_SCM_PRETEND_VERSION_FOR_BASKERVILLE=1.0.0 \
  ~/borzoi-kit-venv/bin/pip install --no-deps -e <baskerville checkout>
SETUPTOOLS_SCM_PRETEND_VERSION_FOR_BORZOI=1.0.1 \
  ~/borzoi-kit-venv/bin/pip install --no-deps -e <borzoi checkout>
export BORZOI_DIR=<borzoi checkout> BASKERVILLE_DIR=<baskerville checkout>

launch.py puts $BORZOI_DIR/src/scripts on the worker's PYTHONPATH, which is how upstream's own image makes borzoi_sad importable — and the worker refuses to start if it is not importable, rather than reimplementing its writer.

One thing the index cannot tell you, and it bites this stack

The kit pins tensorflow==2.15.1. On aarch64 that wheel carries no code: its RECORD is six dist-info lines, and

Requires-Dist: tensorflow-cpu-aws ==2.15.1 ; platform_system == "Linux" and (platform_machine == "aarch64")

hands the real library to a different distribution of the same version. With --no-deps that redirect is never followed, so pip list shows tensorflow 2.15.1 while import tensorflow raises ModuleNotFoundError. The lock therefore keeps the pin and adds tensorflow-cpu-aws==2.15.1 beside it. Check what a wheel contains, not that it exists.

Fetch and verify the weights

tu remote setup borzoi_calico --only weights

Two files, both declared with a size and a sha256 in provider.toml and verified before the worker starts:

file bytes
f0/model0_best.h5 744,112,468
hg38.ml.fa 3,091,663,428

The genome is not the README's UCSC hg38.fa: upstream's download_models.sh:63-67 fetches hg38.ml.fa.gz from its own bucket, and that is what -f defaults to. Contig names must carry the chr prefix.

Start the server and call it

python -m tooluniverse.remote.borzoi_calico.borzoi_calico_tool
tu doctor --forward http://127.0.0.1:8061/mcp --json

{"variants": [{"chrom": "chr1", "pos": 43110773, "ref": "G", "alt": "A"}],
 "top_tracks": 20}

SNVs only, GRCh38, 1-based. A bare 1 is normalised to chr1 in both the tool and the worker — baskerville.vcf.SNP prepends the prefix itself (vcf.py:672-675), so two callers who disagreed about it would otherwise get rows that look like different variants.

Bounds, and what happens at them

bound value at the bound
variants at most 4 refused with the count and the per-variant cost named
alleles ACGT, SNVs only refused by name; indels are not scored
pos integer ≥ 1 refused by name
top_tracks 1–100 refused; the full 6,385-track vectors stay in sad.h5
concurrency one request at a time a second request queues on the worker's lock

Reproducing the equivalence gate

# stock, through the kit's own entry point
cd <kit>/borzoi
PATH=~/borzoi-kit-venv/bin:$BORZOI_DIR/src/scripts:$PATH \
  bash run.sh sad --mode off -f <weights>/hg38.ml.fa -o /tmp/off \
  --rc --stats SAD,logSAD,D2,logD2 -t $BORZOI_DIR/examples/targets_human.txt -u \
  $BORZOI_DIR/examples/params.json <weights>/f0/model0_best.h5 one.vcf

Then the same variant through the worker, and compare dataset by dataset — not as bytes. HDF5 allocation order and the kit's chunked writer differ legitimately between two runs that agree on every number, so a byte diff produces a false negative. Compare each dataset's name, shape, dtype and values exactly: SAD, logSAD, D2, logD2 at (n, 6385) float16, their four (6385, 279) percentile tables, and snp, chr, pos, ref_allele, alt_allele, percentiles, target_ids, target_labels.

Two upstream details this provider preserves on purpose

Stock's sum_length branch is dead. borzoi_sad.py:159 splits --stats into a list; :210 then tests sum_length = (options.sad_stats == 'SAD'), a list against a string, which is always False. So upstream never calls build_sad() and always writes with write_snp_len, for every --stats value including --stats SAD alone. The worker reproduces that rather than correcting it, because its whole claim is that it answers what stock answers.

--shifts is a list by the time it reaches baskerville. :158 converts it, and seqnn.build_ensemble takes len() of it. Passing the bare int raises — which is the good case: a signature that accepted it would have scored every variant without the reverse-complement average the pinned command asks for.

Neither of these was caught by comparing outputs. The first is invisible to the gate because the pinned stats make the bug and a "fixed" version agree; the second failed loudly. Read the stock script; do not infer it from its own output.

What the Platform evidence covers, and what it does not

This implementation was added after the authenticated 2026-08-16 Platform matrix and was not part of it. That matrix imported each implementation it covered as unpublished owner drafts and invoked them through /expert-sessions/{id}/test; this provider was imported as no such draft and invoked through no such route. It has not been exercised through the Platform at all: the evidence above is local, on one card, through the loopback endpoint. Public publication, independent-caller authorization/isolation, saturation, and persistent supervision are untested.

Authorize once, then share with one short command

Run once per machine, and again after key rotation:

tu remote login

Then each private share is:

tu remote share borzoi_calico

Override the defaults only when needed:

tu remote share borzoi_calico --name my-borzoi-calico-remote --workers 1

Sharing automatically starts the browser flow when the key is missing, expired, or revoked. A malformed or revoked explicit TOOLUNIVERSE_SERVICE_KEY fails fast instead of being silently replaced. Use tu remote logout to remove only the locally stored key.

Check discovery and relay privately

tu doctor --forward http://127.0.0.1:8061/mcp --json
tu serve --share --forward http://127.0.0.1:8061/mcp --name validation-borzoi-calico --workers 1

Run a verified example

The Streamable HTTP endpoint is http://127.0.0.1:8061/mcp. In a second activated shell run:

Operation: run_borzoi_calico_sad

{"variants":[{"chrom":"chr1","pos":43110773,"ref":"G","alt":"A"}],"top_tracks":5}

Invoke the example through the live local MCP endpoint:

python - <<'PY'
import asyncio
import json
from fastmcp import Client

async def main():
    arguments = json.loads('''{"variants":[{"chrom":"chr1","pos":43110773,"ref":"G","alt":"A"}],"top_tracks":5}''')
    async with Client("http://127.0.0.1:8061/mcp") as client:
        result = await client.call_tool("run_borzoi_calico_sad", arguments)
        print(result)

asyncio.run(main())
PY

Expected success shape: implementation, n_variants, n_tracks 6385, stats, a variants array whose top_tracks carry all four statistics, and sad_h5. Check scientific meaning, finite values, output bounds, invalid-input behaviour, and the absence of paths, secrets and traces.

Troubleshooting

  • import tensorflow fails after a clean install. The aarch64 tensorflow==2.15.1 wheel is a shim; install tensorflow-cpu-aws==2.15.1 (the lock already does).
  • The worker refuses naming BORZOI_DIR or borzoi_sad.py. The checkout is missing or src/scripts is not where it expects. The worker imports stock's writer and will not proceed without it.
  • The worker refuses naming a non-writable weights directory. pysam needs to write the .fai beside the genome.
  • Unable to synchronously create dataset (name already exists). Something passed the comma-joined --stats string where stock's helpers expect the split list: for sad_stat in sad_stats then iterates characters and dies on the second S.
  • A request takes twenty minutes. Four variants at ~255 s each, on the CPU. That is this provider.