Three models, all local, failing in different directions. Pick by what the material is, not by which is "better".
| MuScriptor medium | Basic Pitch | SwiftF0 | |
|---|---|---|---|
| what it hears | full mix, several instruments | polyphony, one track | one line — the loudest |
| output | labelled MIDI track per instrument + GM drums | one untitled track, tonal only | pitch + confidence per hop |
| drums | yes, GM map (36 kick / 38 snare / 39 clap / 42 hat) | none | none |
| tempo / key | detected, accurate | placeholder 120 BPM | none — it is a tracker |
| velocity | all notes 100 | real, 30 values (36–72) | none |
| licence | code MIT, weights CC BY-NC 4.0 | Apache-2.0, ours to ship | CC-BY-4.0 (lars76), converted by us |
| speed | 52 s for 3:12 of audio | 35 s for the same | 1 s for the same (~190x RT) |
| where | uvx muscriptor, weights cached |
uvx basic-pitch |
~/Music/1music/swiftf0-coreml/ |
Pick by material:
- a band, a track, drums → MuScriptor. It is the only one that separates instruments and gets the grid right.
- something that ships → Basic Pitch. The NC weights bar MuScriptor from a product, and only from a product: transcribing our own recording for our own film is exactly the case the licence allows. (I read that bar wider than it is written once and skipped the better model for no reason.)
- one melody — a voice, a flute, a solo line → SwiftF0. It does not guess a chord, it follows a line, and it is two orders of magnitude faster than the other two.
The commands
# MuScriptor — needs a HuggingFace token, weights are gated
export HF_TOKEN=hf_...
uvx muscriptor transcribe mix.wav --output out.mid --detect-tempo best-effort
uvx muscriptor transcribe mix.wav --format sheets --output score/ # +MusicXML, PDF, tabs
uvx muscriptor list-instruments # names for --instruments
uvx muscriptor serve # local web UI
# Basic Pitch
mkdir -p bp_out
uvx --python 3.11 --with "setuptools<81" --from basic-pitch basic-pitch bp_out mix.wav
# SwiftF0 — одна линия, наша конвертация; тулы живут в video-analyzer
python3 scripts/pitch_line.py solo.wav --notes # ноты с временем
python3 scripts/pitch_line.py solo.wav --theme # строка --chords
python3 scripts/midi_theme.py out.mid --windows 8 # то же из любого MIDI
pitch_line.py carries the preprocessing contract SwiftF0: the ONNX graph contains an
STFT node that no converter picks up, so the model is split and we compute the STFT
ourselves — 16 kHz mono, 384 padding zeros on both sides (constant, not
reflect), frame 1024 / hop 256, asymmetric Hann, rFFT bins 3..135,
log(x + 1e-8). A deviation in any of these points does not break startup, but
silently corrupts the pitch. The decoder is already inside the model: it outputs pitch_hz и
confidence, there is no need to decode a softmax over them.
Four traps, each one cost a run:
--outputwants a file, not a directory. A trailingout/transcribes fine and then dies withIsADirectoryErrorafter the generation is done. The directory form belongs to--format sheetsonly.- The HF gate is per size. Accepting the licence for
muscriptor-mediumleavessmallandlargeat 403. Accept each on its own model page. - Basic Pitch needs
setuptools<81.resampyimportspkg_resources, which setuptools 81 removed, so a clean environment fails withModuleNotFoundError: pkg_resources. Plain--with setuptoolsdoes not fix it — the pin is the fix. - Basic Pitch will not create its output directory.
mkdir -pfirst, or🚨 bp_out is not a directory.
Model sizes and where the compute goes
Measured here, M5 / 32 GB, medium, the same 40 s excerpt. No conversion to
Core ML or ONNX is needed on Apple Silicon: the package checks
torch.backends.mps.is_available() and platform.machine() == "arm64"
(accelerator.py:25) and moves the plain PyTorch weights to Metal itself.
| configuration | generate | × realtime | a 4:49 track |
|---|---|---|---|
| MPS fp16 (default) | 24.1 s | 0.60× | ~2.9 min |
| CPU fp16 | 66.5 s | 1.66× | ~8 min |
| CPU fp32 | 63.1 s | 1.58× | ~7.6 min |
MPS fp16 -b 8 --no-prelude-forcing |
6.6 s | 0.16× | ~48 s |
- A GPU is not required. CPU is 2.8× slower, not orders of magnitude. On CPU, fp32 is marginally faster than fp16 — half precision is emulated there.
- MPS fp16 and CPU return near-identical MIDI (206 notes against 207 on the lead), so half precision on Metal costs nothing.
--batch-size 8buys 3.7× and eats the parts. Drums survive intact (kick 52, snare 22, clap 4, identical), hats lose 7%, and the lead collapses from 206 notes to 84 — without prelude-forcing a chunk cannot see the previous one. Use it to grab a grid and a rhythm section; never to lift a part.- Sizes:
small103M (CPU),medium307M (default),large1.4B (their docs ask for a GPU; untested here — the gate was not accepted for it). --dtype,--device,--model,--instrumentsall exist;--instrumentsforbids everything not listed, which is the cheap way to stop a synth being decoded as three overlapping guitars.
What the numbers are worth
Measured here against a by-hand teardown of Gesaffelstein "Hate or Glory"
(40 s of the main section; the hand analysis is demos8/ANALYSIS.md):
| by hand | MuScriptor | |
|---|---|---|
| tempo | 102.985 BPM | 102.994 (librosa said 103.36 and drifted a beat by bar 88) |
| key | G minor | G / D / A# — G minor |
| bass | G1, 48.6 Hz | G1, 158 notes |
| kick | four-on-the-floor | 51 consecutive beats, 0 missing, median offset −1.2 ms, max 19.4 ms |
That grid accuracy is the part worth having: a transcription with the right notes on a wrong grid cannot be rebuilt from. Their published onset F1 is 60.4, so expect roughly this on rhythmic material and worse on anything loose.
Three things not to believe, all three measured:
- Velocity. Every note comes out at 100 — all 657 of them. The model has no velocity representation, so dynamics must be written afterwards or the part plays like a machine. Basic Pitch does give real velocities.
- Durations. The measured bass is a 540 ms stab (0.93 of a beat); the model returned a median of 0.26 and shredded the pedal note into re-attacks. It holds no sustain. Onsets and grid, yes; length and articulation, measure from the signal.
- Instrument names. A synth lead came back as
prog=29 distorted electric guitar. The part is right, the GM label is a guess — reassign it yourself.
Also from their docs: rubato recordings notate significantly worse. Beat-driven references transcribe well, ambient is not worth the run.
One recording, three models — and what can be asserted from it
Measured here, 23 Sep 2026. A street piano recorded with a DJI microphone in the middle of Navy Pier: wind, cars, voices. 3:12 of audio, three models independently:
| notes | time | chords across eight windows | |
|---|---|---|---|
| SwiftF0 | 325 (82% confident frames) | 1 с | Am Am Cm Dm Bm Dm Am E |
| Basic Pitch | 547 | 35 с | Am Am C Dm F Dm Bm Em |
| MuScriptor | 376, track acoustic piano |
52 с | Am Bm Dm Bm Am C F |
All three converge on Am. The beginning and the fourth window match for two of the three. The middle is different for each — so nothing should be asserted about the middle.
Hence the practical rule for a noisy recording: harmony is recoverable, the motif is not. A pitch-class histogram weighted by duration averages out false notes and gives a stable answer; the note sequence on such material is not reproducible (for Basic Pitch the most frequent "figure" of the top voice is one note three times in a row, and the range 36…95 includes octave jumps that a human did not play). Such a theme should be inherited as a key and a chord progression, not as a melody.
The check without which it silently lies
A 440 Hz sine is run first, before any material. The answer is known in advance, and only for that reason is it visible that the instrument works.
Spelling pitch_line.py produced two defects, neither of which was visible in the output:
decoding softmax on top of already-computed hertz (the model decodes internally) and
closing a note only on a pitch change — a steady tone produced no notes
at all. Both times the tool answered "no confident pitch found," and without
controls the report would have been "the street piano is too noisy for a monophonic
tracker" — a confident lie about the material.
What a healthy response looks like: # 3.0s · нот 1 · уверенных кадров 100%,
0.00 A4 2.99s, 437.5 Hz versus 440 — one logarithmic bin.
The same rule for any of the three models: don't trust a zero from an instrument you haven't seen give a non-zero.
Into film music, not just into instruments
A film's theme can be derived from what a person actually played on screen:
python3 scripts/pitch_line.py piano.wav --theme # → --chords "Am,Am,Cm,..."
python3 scripts/score_gen.py --plan film.otio --out score/ \
--key A --chords "Am,Am,Cm,Dm,Bm,Dm,Am,E" --bpm 92,104,96 \
--silence "1.0-3.6" # там, где играет живое, партитуры нет
--silence makes a hole in the score rather than lowering the volume: inside the window
−91 dB versus −13.6 at full strength. Music that ducks under a live instrument
is audible; a hole is not.
Into our instruments
The drum track already uses the GM map, which is what SuperDuper Drum reads
with Note Map on 0 Auto (36/35 kick, 38/40 snare, 39 clap, 42/44 hat) — the
MIDI drops in with no remapping. Tonal tracks go to Wave or Kubyz, with the GM
program ignored.
Build the arrangement by writing the .rpp directly rather than pushing items
through the REAPER bridge — see the project CLAUDE.md, and demos5/build_rpp.py
for a working generator. For notes there is one more route: the bridge's generic
passthrough via reaper_ipc.py, which inserts notes in bulk without inlining
thousands of them into tool arguments.
Check before you trust
uvx --with mido python scripts/midinspect.py out.mid
uvx --with mido python scripts/gridcheck.py out.mid drums 36
midinspect prints per-track note counts, ranges, pitch-class histograms and
duration medians — enough to see the key, spot a doubled octave and catch a part
that came back empty. gridcheck answers the question a note list cannot:
whether the onsets sit on the beat grid, with the deviation in ms and every
beat that was missed. An empty result exits 2 and says nothing was measured,
because "no findings" and "nothing checked" must not look the same.
Not measured here
Reported but unverified in this repo, in case a run needs more than the three above: NeuralNote (Apache-2.0, VST3/AU wrapper around Basic Pitch, actively developed — 2026-09-16 — so the way to get this inside REAPER without a CLI), and MT3 (Google, multi-instrument, JAX/T5X, awkward to run). Mirelo's hosted Audio-to-MIDI Pro is the same lineage as MuScriptor with a more accurate closed model and an API, at https://mirelo.ai/models/audio-to-midi.