Mirai is built by Mirai Labs and positioned as "the future of on-device AI." Its core idea is that the next generation of AI products will be built around continuous interaction, rather than isolated prompts and chat templates; this experience is only possible once latency disappears.
I. Main Products and Capabilities Mirai Labs has built uzu—a Rust LLM engine for iPhone, iPad, and Mac. It also provides 4-bit/8-bit Qwen, LFM, and Muse-Glimmer models, benchmarked against MLX and llama.cpp. It officially says it provides "the models, runtime, and infrastructure that make on-device AI interactive, ambient, and continuous."
II. Product Philosophy: From an App Grid to a Single Interface It argues that over the past decade every service has had its own separate app, and users have to switch among dozens of interfaces to get one thing done (banking, calendar, files, messages, etc.). The next decade will collapse the app grid into a single interface: one instruction, multiple services, zero app launches. The assistant handles all tasks in a single interaction, privately on the device locally, for example handling Chase, calendar, files, Uber, and more at the same time.
III. Latency and Performance Claims It proposes that "1000 t/s is the tipping point where a model becomes an interface." It believes the user experience collapses when the model is too slow; today's AI interfaces are just chat boxes, where users enter a request and wait, and people do not like loading states longer than 200 ms. The user sees only 1 reply, while internally the model has to generate more than 100 steps, including parsing, validation, tool calls, reasoning, and more. It argues AI should be so fast that people do not notice it at all.
IV. Technical Approach: Full-Stack In-House Development It states that achieving 1000 t/s requires a different execution stack. Interactive AI is not a single response, but a continuous loop: parsing, validation, ranking, rendering, with every step having a latency budget, and any step timing out breaks the experience. To this end, they built every layer from scratch. It also explains why cloud models cannot simply be run directly on-device: on-device is not a shrunken version of the cloud, but an entirely different system. Performance comes from full-stack co-design. On-device, every request is handled separately, with no batching and no parallelism. Mirai co-designs the model and runtime on Apple Silicon for this constraint, with the goals of minimizing memory footprint, maximizing arithmetic intensity, and maximizing neural accelerator utilization.
V. Positioning and Vision It positions Mirai as a frontier lab for on-device AI: the chips are already in place, 2 billion devices are waiting, software is the remaining unsolved problem, and that is exactly what they aim to solve.
VI. Limitations and Information Not Provided The above is the content stated on the official page. Specific pricing and payment methods, model license agreements, download and installation commands, supported specific OS versions, API interfaces, and specific evaluation data values are not yet provided in the materials and should be subject to subsequent official disclosures.
