In today's digital age, artificial intelligence (AI) has become a key force driving technological progress and innovation. Apple, as one of the giants in the global technology field, has been continuously exploring and innovating to provide a more intelligent and personalized user experience. At the 2024 WWDC conference, Apple unveiled its latest masterpiece—Apple Intelligence. This is a brand-new personal intelligence system designed to completely transform the way users interact with devices.
What is Apple Intelligence
Apple Intelligence is a personal intelligence system developed by Apple that is integrated across the iPhone, iPad, and Mac platforms, and was announced at the WWDC conference (Apple Worldwide Developers Conference) on June 11, 2024, Beijing time. This AI system provides users with useful and relevant services and experiences by combining powerful generative models with personal context. The core of Apple Intelligence lies in its advanced AI technology and emphasis on privacy. It can not only understand users' needs but also predict users' intentions, while protecting user privacy and providing more personalized services.
Core features of Apple Intelligence
AI-powered writing tools
Writing Tools is a system-level feature of Apple Intelligence that helps users rewrite, proofread, and summarize text in various applications. Whether drafting emails, polishing blog posts, or organizing class notes, these tools can make users more confident when writing. Users can adjust the tone of the text, check for grammatical errors, and even get concise summaries of the text content.
Genmoji and Image Playground
Apple Intelligence also brings fun and creativity to users' images and emoji. Through the new Image Playground, users can generate fun images in three styles: animation, illustration, or sketch. Users can create custom emoji, called "Genmoji," that perfectly capture moments and unique expressions. In addition, using Image Wand, users can also turn sketches into polished images.
Improved photo and video experience
The Photos app has also been enhanced, with new natural language search that allows users to easily find specific moments in videos. The new Clean Up tool can remove distractions from the background without changing the subject. With the "Memory" feature, users can create custom stories simply by describing them.
Privacy and security
Apple Intelligence sets a new privacy standard in the AI field. First, it has small models that run entirely on-device to process requests locally. When more complex tasks require cloud processing, they have a new private cloud computing feature, while also ensuring that user data is not retained or exposed.
A comprehensive upgrade for Siri
Siri has received a comprehensive upgrade through Apple Intelligence. Now, Siri understands users better, can follow users' train of thought, maintain context between requests, and even allow users to type instead of speak. Siri can also perform hundreds of new actions in applications, such as getting book recommendations from friends or checking parents' flight status.
ChatGPT integration
Apple has also integrated OpenAI's ChatGPT on its platforms. Siri can use ChatGPT's expertise to answer questions with the user's permission. ChatGPT will assist in system-wide writing tools, generating content and images. ChatGPT users can also connect their accounts to access paid services.
Technical details of Apple Intelligence
Apple Intelligence is built on a series of generative models created by Apple, including on-device and server foundation models, diffusion models for image generation, and coding models. In addition, Apple Intelligence can also invoke third-party models, such as ChatGPT, as needed to handle more complex requests.
On-device model
The on-device model has about 3 billion parameters and a 49K vocabulary, using low-bit quantization and grouped query attention techniques to improve speed and efficiency. On iPhone 15 Pro, the model achieved a time to first token latency of 0.6 milliseconds per prompt token and a generation rate of 30 tokens per second.
Server-side model
The server-side model has a 100K vocabulary and can use private cloud computing to handle more complex tasks while ensuring privacy and security. The model uses advanced techniques, such as speculative decoding and context pruning, to improve performance. Built on a hardened subset of the iOS foundation, it ensures the privacy of user data through strong encryption and a secure boot process.
Training and optimization
Apple's models are trained on carefully curated datasets that do not contain any personal user data. The training data includes licensed data, publicly available data collected by AppleBot, and synthetic data. After training, Apple uses novel algorithms such as rejection sampling fine-tuning and reinforcement learning from human feedback to improve the models' ability to follow instructions.
Apple has implemented a series of cutting-edge technologies to ensure optimal performance and efficiency on mobile devices. By using methods such as grouped query attention, shared embedding tables, low-bit palletization, and efficient key-value cache updates, Apple has successfully created highly compressed models that maintain quality while meeting the memory, power, and performance constraints of mobile devices.
Low-Rank Adaptation (LoRA)
Unlike general-purpose models such as Google's Gemini Nano and Microsoft's Phi, Apple's models are fine-tuned for everyday activities such as summarization, email replies, and proofreading by using a technique called "low-rank adaptation." This technique involves inserting small neural network modules into the various layers of a pre-trained model. This allows the model to adapt to different tasks while retaining its general knowledge. Importantly, these adapters can be dynamically loaded and swapped, allowing the foundation model to be specialized for the task at hand.
How to use Apple Intelligence
Apple Intelligence is available to users for free, and the beta version will be released this fall as a built-in feature of iOS 18, iPadOS 18, and macOS Sequoia, supporting only English (US). Some features, software platforms, and other language support will be rolled out next year. Apple Intelligence will only support iPhone 15 Pro, iPhone 15 Pro Max, and iPad and Mac devices with M1 or later chips, and Siri and the device language must be set to English (US).