Extentos MCP Server is an MCP (Model Context Protocol) server designed to let AI coding agents (such as Claude Code, Cursor, Cline) add smart glasses capabilities to Android and iOS apps. Its core purpose is to help developers complete the full workflow of discovery, scaffolding, validation, and simulation before touching real hardware.
Main capabilities include:
- Discovery and guidance: getPlatformInfo provides an SDK capability catalog, version/artifact information, and platform constraints; getCapabilityGuide provides feature-by-feature invocation forms and caveats for Kotlin and Swift; getCodeExample provides canonical end-to-end patterns (voice Q&A, photo description, live transcription UI, etc.); getMigrationGuide provides a mapping table and switchover plan for migrating from direct Meta DAT usage to Extentos.
- Scaffolding and configuration: generateConnectionModule scaffolds the connection UI, manifest, and platform configuration into the app; getConnectionPageConfig, setConnectionPageConfig, regenerateConnectionPageFile, adoptConnectionPageFile manage connection page configuration; getAssistantConfig, setAssistantConfig manage voice assistant configuration (model, voice, memory, wake sound); listProjectSounds, addProjectSound manage project custom sounds.
- Validation and inspection: inspectIntegration reads the current state of the integration in the repository; validateIntegration performs deterministic checks on the integration (manifest, configuration, wiring); getPermissions retrieves the exact permission set required for each platform; getCredentialStatus views configured credentials; setCredential stores credentials; getProductionChecklist provides the path from simulator to real glasses; getCredentialGuide explains how to obtain and configure Meta DAT credentials.
- Simulator and endto-end testing: createSimulatorSession creates a browser simulator session; ensureSimulatorBrowser opens/attaches the simulator browser tab; completeAuthLink completes the device code login flow; getEventLog reads the session event log; injectTranscript injects voice transcription; injectAssistantUtterance drives the voice assistant end to end; assertToolCalled waits for a matching assistant tool call event; getSimulatorStatus gets real-time session status; setSimVideo pipes a test video into the simulated camera; setSimDevice switches the simulated glasses model; getDisplayState reads the rendered display tree and interactive element IDs; injectInput drives display input; injectHardwareButton simulates the hardware capture button; shutter triggers the camera shutter.
- Other: getGatewayUsage gets AI gateway usage and metering; getProjectAnalytics gets usage analytics for the project's glasses integration; getVoiceCommandGuidance provides implementation guidance for voice triggers and wake phrases; searchDocs searches bundled versioned documentation topics.
How to use: apps depend on the native SDK (Android uses com.extentos:glasses, Maven Central; iOS uses a Swift package). Code subscribes to capability primitives from its own handler classes, such as glasses.audio.transcriptions(), glasses.camera.capturePhoto(), glasses.audio.speak(). The simulator is the same app running on a different substrate: it uses WebSocket transport instead of Bluetooth, so behavior validated by the agent can carry over to hardware. Discovery, validation, and guidance tools can be used anonymously; creating a browser simulator session and scaffolding require a free account, and the server guides the agent (and user) through device code login when needed.
Use cases: adding smart glasses functionality to an Android or iOS app, especially when using Meta smart glasses (Ray-Ban Meta, Oakley Meta, Meta Ray-Ban Display). Community integration examples include EgoFlow (an egocentric video capture platform whose extentos branch drives glasses connectivity and live video through the Extentos Android SDK, and has been run end-to-end on real Ray-Ban Meta glasses) and the voice-triggered capture POC (an MIT-licensed iOS proof of concept that registers a wake phrase to trigger glasses.camera.capturePhoto, driven headlessly in a browser simulator via an MCP tool).
Limitations: This package is pre-1.0, and the tool interface may evolve between minor versions. Node 20+ is required. For security reports, see SECURITY.md, and do not publicly submit vulnerability issues.
