Alumnium

Alumnium is a collection of libraries and tools for natural language end-to-end testing of web and mobile applications, with the goal of enabling AI agents and engineers to describe test intent in human-readable language, while Alumnium handles the underlying UI interaction details.

Main components Alumnium includes the following parts:

• MCP server, for agent testing;
• Java, Python, and TypeScript client libraries, for writing tests;
• Test runner, for executing Markdown scenarios locally or in CI;
• Server, for centrally handling model communication and caching.

How to use There are two main integration paths:

1. Connect to an MCP-compatible coding agent via MCP. First install the Alumnium binary using an installation script or package manager, then choose a supported AI provider and configure credentials if needed, and then connect Alumnium to an MCP-compatible coding agent. After that, describe user flows and expected results in natural language, and the coding agent drives and verifies the application through Alumnium; you can also have the agent save successful scenarios as Markdown, and then repeatedly execute them locally or in CI with the Alumnium test runner.
2. Use the client libraries in an existing test suite. When initializing Alumnium, pass in your existing Selenium, Playwright, or Appium driver, then add natural language actions, checks, and data retrieval alongside your framework code, gradually introducing AI without having to replace your existing test stack.

Supported frameworks and platforms

• Web automation: Selenium or Playwright.
• iOS and Android mobile testing: Appium or WebdriverIO; Maestro is also available through the MCP server.
• Client libraries support Java, Python, and TypeScript.
• Alumnium is distributed as a single-file binary, about 100 MB in size, and supports macOS, Windows, and Linux.
• Alumnium Server supports programmatic control via an HTTP API.

Model support Alumnium is also designed for smaller, lower-cost models. You can choose Anthropic, OpenAI, Google, xAI, Meta, DeepSeek, MistSupported hosted models from providers such as ral, and supported local models can also be run via Ollama. Changing models does not require changing the test intent, and Selenium, Playwright, Appium, or Maestro can still be used underneath. For scenarios that need to keep application data within your own infrastructure, local models can be run via Ollama.

Core capabilities Alumnium provides a set of high-level testing tools: plan actions based on natural language goals and execute them in the application; check whether the application is in the expected state and explain the result; retrieve requested data or state from the application; wait for the application to reach a condition described in natural language; summarize UI changes after an action; use screenshots when checks or retrieval require visual information; find elements based on natural language descriptions; focus on specific areas of the page for more precise interaction.

Under the hood, Alumnium mainly uses text-based representations to understand the application structure and state, adding visual capabilities only when a task requires visual information. It converts the platform's accessibility tree into focused XML, providing structured context to the model. It supports browser workflows such as tab navigation, framed pages, Shadow DOM, file uploads, persistent profiles, as well as automatic waiting and retries, screenshots and video, executing JavaScript, history navigation, slider operations, headless browsers, cookie assignment, browser permissions, Playwright trace, custom HTTP headers, custom User-Agent, and other capabilities; the specific level of support varies by automation framework and platform, and the support matrix in the documentation should be consulted.

Caching and performance Alumnium uses multi-level caching: it caches model responses and element decisions to reduce repeated large model requests in subsequent test runs. Element caching can remain effective across dynamic identifiers or minor UI changes, while larger changes may still trigger new model requests. Its prompt structure also takes provider-side caching into account. It is officially stated that in the WebVoyager browser agent benchmark, Claude Code with Alumnium MCP achieved a 98.5% pass rate across 610 tasks.

Markdown test runner and extensibility Alumnium includes a preview test runner for repeatable execution of Markdown scenarios: scenarios can be written manually or generated by a coding agent with Alumnium MCP, reviewed and saved in the repository, and then run with alumnium test <filename>.md. The runner records tool calls for reuse; when the cache is warmed and the flow is unchanged, replay may require no new LLM requests. When cached steps no longer match the UI, Alumnium re-parses first and then reports failure. High-level Markdown scenarios can be reused across configured Web, iOS, and Android sessions. The runner can use native agent SDKs, currently supporting the Claude Agent SDK.

Scale and Integration Alumnium Server runs as a single-file binary on the user's infrastructure, handling model requests, caching, and test execution for multiple agents and test runners. The cache can be stored on the local file system or remotely (preview), so team members and CI jobs do not each have to repeat the same model requests. The test runner supports parallel execution on a single machine and can also be configured to shard large test suites across multiple machines. Alumnium can connect to cloud services that provide real devices and browsers, such as Selenium Grid, Sauce Labs, and TestMu. When tracing is enabled, telemetry data can be exported to any OpenTelemetry-compatible collector.

Project Characteristics and License Alumnium is open source under the MIT License, allowing personal and commercial use, and development, issues, and pull requests all take place in public. Team members have led open source projects such as Selenium and date-fns. Users can ask questions, provide feedback, or discuss contributions through the Discord community.

Use Cases

• End-to-end testing where you want to express test intent in natural language and reduce maintenance of selectors and explicit waits.
• Existing Selenium, Playwright, and Appium test stacks where you want to gradually introduce AI without replacing the existing framework.
• Agentic testing workflows that need coding agents to drive and verify Web or mobile applications.
• Reusing high-level test instructions across Web, iOS, and Android.

Limitations and Considerations Alumnium makes requests to model providers on its own, so it usually requires a valid API key or other credentials for the chosen provider; local Ollama models do not require this. The Markdown test runner and remote cache are currently in preview. Specific capabilities vary across automation frameworks and platforms, so the support matrix in the official documentation should be consulted. The above content does not cover specific fees or pricing information, and related materials are not currently available.