org-ai is an Emacs minor mode that provides access to generative AI models for org-mode. The models it currently supports include the OpenAI API (ChatGPT, DALL-E, other text models), with the option to run through the Azure API instead of OpenAI, as well as Stable Diffusion through stable-diffusion-webui.
Main capabilities: In an org-mode buffer, you can use ChatGPT to generate text, with full control over the system prompt and user prompt. It supports voice input and output, allowing you to converse with the AI. You can use text prompts to generate images and image variations through Stable Diffusion or DALL-E. org-ai also provides various commands for use outside org-mode, allowing you to prompt selected text or multiple files.
Usage: Use the #+begin_ai...#+end_ai special block to mark input to the AI model (and also output for ChatGPT). It can be used for AI chat, text completion, and text-to-image generation. After creating a block, press C-c C-c to create a conversation, and press C-g to interrupt a running request. Use the :image keyword to generate images, using DALL-E-3 by default, and you can control image generation through keywords such as :size, :model, :quality, :style, :n. You can modify the system prompt and other parameters; the system prompt is injected before the user input and is used to "guide" the model to answer in a particular style. When generating images with the :image flag, the images are displayed inline below the ai block and stored in org-ai-image-directory (default ~/org/org-ai-images/). You can also use voice input transcription for input; press C-c r to start recording (speech recognition must be configured). Voice output can be enabled with org-ai-talk-output-enable. Within #+begin_ai...#+end_ai, you can use C-c <backspace> (org-ai-kill-region-at-point) to delete the chat section under the cursor, org-ai-mark-region-at-point to mark the region at the cursor, and org-ai-mark-last-region to mark the last chat section. You can add a language major mode name after _ai to apply syntax highlighting, for example #+begin_ai markdown. For markdown, you can set (setq markdown-fontify-code-blocks-natively t) to correctly highlight code in backticks. Automatic jumping to the end of the block is enabled by default, and automatic filling can be enabled with (setq org-ai-auto-fill t).
Block options: For ChatGPT: :max-tokens, :temperature, :top-p, :frequency-penalty, :presence-penalty, :sys-everywhere. You can use org file-scoped properties or an org heading property drawer to make blocks inherit settings. Custom variables include org-ai-default-chat-model (default "gpt-4o-mini"), org-ai-default-max-tokens, org-ai-default-chat-system-prompt (default "You are a helpful assistant inside Emacs."), org-ai-default-inject-sys-prompt-for-all-messages. For DALL-E: :image, :size (default 256x256, can be 512x512 or 1024x1024), :n. Custom variable org-ai-image-directory. Other text models: :completion, :model. Custom variable org-ai-default-completion-model (default "text-davinci-003"). Image variations: The org-ai-image-variation command prompts for the image file path, size, and number, generates images, and inserts links. The input image must be square and smaller than 4MB, and the curl command-line tool is required.
Global commands: After enabling org-ai-global-mode, the prefix C-c M-a is bound to multiple commands. The org-ai-on-project buffer allowsrunning commands on files in a project, or only on selected text. You can ask the AI to modify or refactor code, generate a diff, and choose whether to apply it.
Installation: org-ai is on Melpa:https://melpa.org/#/org-ai。添加 After Melpa to the package archive, it can be installed using package-install. You can also manually check out the repository and use use-package or require.
API keys: You can set the API token directly in the configuration, or use auth-source to retrieve it from ~/authinfo.gpg. You can switch to Azure via custom variables such as org-ai-azure-openai-api-base. You can also use perplexity.ai or Anthropic / Claude, with the corresponding service and authentication configuration.
Speech input/output settings: Speech input uses whisper.el and ffmpeg, and has been tested on macOS and Linux. Windows users need to test it themselves.
Stable Diffusion settings: Used via stable-diffusion-webui. img2img can be used by marking org-mode images with #+name and referencing them with :image-ref. M-x org-ai-sd-clip and org-ai-sd-deepdanbooru can guess prompts.
Local LLM: Use oobabooga/text-generation-webui to set up a local LLM. After starting the API server, add the :local key in the org-ai block, and requests will be sent to the local API server instead of the OpenAI API.
Noweb support: Noweb references are supported, and the expanded document can be viewed via org-ai-expand-block.
Limitations: An OpenAI account and API token (or corresponding service credentials) are required. Free-tier usage limits are not currently provided. perplexity.ai currently does not provide citations/links via its API. Other limitations such as model versions and API versions are detailed in the documentation.
