ios-agent is an open-source project that lets AI agents operate an iOS Simulator or a real iPhone, using Apple's public XCTest/XCUIAutomation frameworks and WebDriverAgent to drive the device, with support for connecting over cable or Wi-Fi. Its core design is that every operation returns the screen state actually generated after the operation, along with a screen_changed flag, so the agent can immediately discover that a tap did not take effect or that entered text did not land; even on a real iPhone, a no-op also returns screen_changed=False. When entering text, it reads back from the input field to verify; an unsuccessful entry returns ok: false along with the actual content displayed in the field. The agent locates elements via element references (such as e2) rather than coordinates; if a screen change invalidates a reference, the host re-finds it by element identity through a six-layer mechanism, and if the reference points to a different element, execution is refused. Toggle operations set a target state rather than switch, and replayed steps avoid duplicate taps via idempotency keys.
In terms of token consumption, the screen is returned as a compact summary rather than accessibility XML; measured on a real third-party screen, 251 raw nodes were compressed into 12 elements, and across thirteen golden flows each step used about 50 to 474 tokens; when the next screen is similar, the operation returns only the changed parts. A 300-row contact list can be returned in a single observation with 438 tokens. ios_find can search the full tree to compensate for summary omissions, preventing compression from permanently hiding controls.
On security, an approval gate is enabled by default: sending, paying, purchasing, deleting, confirming, exiting, and any operation that reaches other people (liking, following, commenting, replying, sharing, inviting, messaging, posting) require approval before they happen; when there is no approver, all are denied. Approval is limited to a single operation; approving a send will not approve a delete. Passwords are read from the host keychain via ios_type_secret and sent directly to the device, without entering the prompt or audit records, and after entry all returned screens substitute [secret]; card numbers (including grouped forms) and email addresses are redacted within the session, and the MCP server, built-in agent, and terminal application all see the redacted screen. Repeated failures or a detected loop will terminate the session. The device selector does not preselect a real phone, and connecting to a real device always requires a single keypress. All defaults can be adjusted via IOS_MCP_* environment variables, .env, or ios-mcp.toml.
There are three ways to use it: the terminal application can stream the model's reasoning process and screen summaries, and display the operation count, device tokens, and cost; manual mode requires no API key; the MCP server provides 31 tools, 5 resources, and the ios_operator prompt, supports stdio or HTTP, and can be connected to MCP clients such as Claude Code, Cursor, Claude Desktop, and VS Code; the Python library IosSession can be called directly. Model providers are configured through LangChain's init_chat_model, supporting Anthropic, OpenAI, Azure OpenAI, Gemini, Vertex AI, Bedrock, Groq, Mistral, and local Ollama; switching requires only two environment variables.
Applicable scenarios include: verifying the effect of code changes in a running app, changing settings, reading answers from an app without an API, and walking through flows on a phone that cannot be handed to a test suite. It is not suitable for Android, other platforms, building, or performance analysis; Flutter canvases and WebViews have no readable tree, and the summary will indicate this and point to a screenshot rather than returning an empty screen; the approval gate does not defend against instructions implanted in the screen; a real iPhone requires a signed runner, and a free Apple ID's provisioning profile is valid for only seven days.
