foundation-models-on-device

FoundationModels: On-device large language models (iOS 26)

Patterns for integrating Apple's on-device language model into apps using the FoundationModels framework. Covers text generation, structured output using @Generable , custom tool calling, and snapshot streaming — all running on-device to support privacy protection and offline use.

When to activate

• Build AI-powered features on-device using Apple Intelligence
• Generate or summarize text without relying on the cloud
• Extract structured data from natural language input
• Implement custom tool calling for domain-specific AI operations
• Stream structured responses for real-time UI updates
• AI that requires privacy protection (data does not leave the device)

Core pattern — Availability check

Before creating a session (Session), always check the model's availability:

struct GenerativeView: View {
    private var model = SystemLanguageModel.default

    var body: some View {
        switch model.availability {
        case .available:
            ContentView()
        case .unavailable(.deviceNotEligible):
            Text("设备不符合 Apple Intelligence 的使用条件")
        case .unavailable(.appleIntelligenceNotEnabled):
            Text("请在设置中启用 Apple Intelligence")
        case .unavailable(.modelNotReady):
            Text("模型正在下载或尚未就绪")
        case .unavailable(let other):
            Text("模型不可用：\(other)")
        }
    }
}

Core pattern — Basic session

// 单轮：每次创建一个新会话
let session = LanguageModelSession()
let response = try await session.respond(to: "去巴黎旅游哪个月份比较好？")
print(response.content)

// 多轮：复用会话以保留对话上下文
let session = LanguageModelSession(instructions: """
    你是一个烹饪助手。
    请根据食材提供食谱建议。
    建议要保持简短且实用。
    """)

let first = try await session.respond(to: "我有鸡肉和米饭")
let followUp = try await session.respond(to: "那素食选择呢？")

Key points of prompt instructions (Instructions):

• Define the model's role ("You are a mentor")
• Specify what to do ("Help extract calendar events")
• Set style preferences ("Answer as briefly as possible")
• Add safety measures ("For dangerous requests, answer 'I cannot help with that'")

Core pattern — Guided generation with @Generable

Generate structured Swift types instead of raw strings:

1. Define a Generable type

@Generable(description: "关于猫的基本个人资料信息")
struct CatProfile {
    var name: String

    @Guide(description: "猫的年龄", .range(0...20))
    var age: Int

    @Guide(description: "关于猫性格的一句话简介")
    var profile: String
}

2. Request structured output

let response = try await session.respond(
    to: "生成一只可爱的待领养小猫",
    generating: CatProfile.self
)

// 直接访问结构化字段
print("名字: \(response.content.name)")
print("年龄: \(response.content.age)")
print("简介: \(response.content.profile)")

Supported @Guide constraints

• .range(0...20) — numeric range
• .count(3) — array element count
• description: — semantic guidance for generation

Core pattern — Tool calling

Allow the model to call custom code to perform domain-specific tasks:

1. Define a tool (Tool)

struct RecipeSearchTool: Tool {
    let name = "recipe_search"
    let description = "搜索匹配给定术语的食谱并返回结果列表。"

    @Generable
    struct Arguments {
        var searchTerm: String
        var numberOfResults: Int
    }

    func call(arguments: Arguments) async throws -> ToolOutput {
        let recipes = await searchRecipes(
            term: arguments.searchTerm,
            limit: arguments.numberOfResults
        )
        return .string(recipes.map { "- \($0.name): \($0.description)" }.joined(separator: "\n"))
    }
}

2. Create a session with tools

let session = LanguageModelSession(tools: [RecipeSearchTool()])
let response = try await session.respond(to: "帮我找一些意面食谱")

3. Handle tool errors

do {
    let answer = try await session.respond(to: "寻找番茄汤的食谱。")
} catch let error as LanguageModelSession.ToolCallError {
    print(error.tool.name)
    if case .databaseIsEmpty = error.underlyingError as? RecipeSearchToolError {
        // 处理特定的工具错误
    }
}

Core pattern — Snapshot streaming

Use the PartiallyGenerated type to stream structured responses for real-time UI:

@Generable
struct TripIdeas {
    @Guide(description: "未来旅行的想法")
    var ideas: [String]
}

let stream = session.streamResponse(
    to: "有哪些令人兴奋的旅行点子？",
    generating: TripIdeas.self
)

for try await partial in stream {
    // partial: TripIdeas.PartiallyGenerated (所有属性均为 Optional)
    print(partial)
}

SwiftUI integration

@State private var partialResult: TripIdeas.PartiallyGenerated?
@State private var errorMessage: String?

var body: some View {
    List {
        ForEach(partialResult?.ideas ?? [], id: \.self) { idea in
            Text(idea)
        }
    }
    .overlay {
        if let errorMessage { Text(errorMessage).foregroundStyle(.red) }
    }
    .task {
        do {
            let stream = session.streamResponse(to: prompt, generating: TripIdeas.self)
            for try await partial in stream {
                partialResult = partial
            }
        } catch {
            errorMessage = error.localizedDescription
        }
    }
}

Key design decisions

Decision	Rationale
On-device execution	Privacy — data does not leave the device; supports offline operation
4,096 token limit	On-device model constraint; chunk large data across sessions
Snapshot streaming (not incremental)	Friendly to structured output; each snapshot is a complete partial state
@Generable macro	Compile-time safety for structured generation; automatically generates PartiallyGenerated type
One request per session	isResponding Prevents concurrent requests; create multiple sessions if needed
response.content(instead of .output）	Correct API — always access the result via the .content property

Best practices

• Always check model.availability before creating a session — handle all unavailable cases
• Use instructions to guide model behavior — they take precedence over prompts
• Before sending a new request, check isResponding — the session processes one request at a time
• Access response.content to get results — not .output
• Split large inputs into chunks — the 4,096 token limit applies to the sum of instructions + prompts + output
• Use @Generable for structured output — stronger guarantees than parsing raw strings
• Use GenerationOptions(temperature:) to adjust creativity (higher is more creative)
• Monitor with Instruments — use Xcode Instruments to analyze request performance

Anti-patterns to avoid

• Creating a session without first checking model.availability create a session in the case of
• Sending input that exceeds the 4,096 token context window
• Attempting concurrent requests on a single session
• Use .output instead of .content to access response data
• In @Generable parsing raw string responses when structured output is available
• Building complex, multi-step logic in a single prompt — split into multiple targeted prompts
• Assuming the model is always available — device eligibility and settings vary

When to use

• On-device text generation for privacy-sensitive apps
• Extracting structured data from user input (forms, natural language commands)
• AI-assisted features that must work offline
• Streaming UI that displays generated content incrementally
• Performing domain-specific AI operations through tool calling (search, calculate, look up)