on-device-ai

On-Device AI with Foundation Models (iOS 26+)

Build intelligent features using Apple's on-device 3B parameter LLM — free, offline, private.

Requirements

iOS 26+ / macOS Tahoe 26+ / iPadOS 26+
Apple Silicon (M1 or newer)
Apple Intelligence enabled on device
import FoundationModels

Core API

Basic Text Generation

import FoundationModels

let session = LanguageModelSession() let response = try await session.respond(to: "Summarize this article in 3 bullets") print(response.content) // String


### Streaming Responses
```swift
let stream = session.streamResponse(to: "Explain quantum computing")
for try await partial in stream {
    print(partial.content) // Incremental text
}

Structured Output with @Generable

@Generable
struct RecipeSuggestion {
    var name: String
    var ingredients: [String]
    var cookingTimeMinutes: Int
    var difficulty: Difficulty

    enum Difficulty: String, Generable {
        case easy, medium, hard
    }
}

let session = LanguageModelSession()
let recipe: RecipeSuggestion = try await session.respond(
    to: "Suggest a quick pasta recipe",
    generating: RecipeSuggestion.self
)

Guided Generation with @Guide

@Guide("Generate a haiku about nature")
struct NatureHaiku {
    @Guide("First line, 5 syllables")
    var line1: String
    @Guide("Second line, 7 syllables")
    var line2: String
    @Guide("Third line, 5 syllables")
    var line3: String
}

Tool Calling

struct WeatherTool: Tool {
    let name = "get_weather"
    let description = "Get current weather for a location"

    struct Input: Codable {
        let location: String
    }

    struct Output: Codable {
        let temperature: Double
        let condition: String
    }

    func call(with input: Input) async throws -> Output {
        // Fetch weather from API
    }
}

let session = LanguageModelSession(tools: [WeatherTool()])
let response = try await session.respond(to: "What's the weather in London?")

Best Practices

Always check LanguageModelSession.isAvailable before use
Provide fallback UI when model is unavailable (older devices, Intelligence disabled)
Use @Generable for structured output instead of parsing raw text
Keep prompts concise — 3B model works best with clear, focused instructions
Use tools for actions that need real data (API calls, database queries)
Stream responses for better perceived performance on longer outputs

When NOT to Use

Complex reasoning tasks (use server-side Claude/GPT instead)
Tasks requiring internet data (model is offline-only)
Image/audio generation (this is text-only)
When user hasn't enabled Apple Intelligence

About this skill