RealChar is an open-source real-time AI character/companion creation and conversation platform, with the slogan "one-stop codebase." It allows users to create, customize, and converse with AI characters in real time, supporting natural and seamless communication with AI on mobile, web, and terminal.
[Main Uses]
- Create and customize your own AI character or companion without coding.
- Have real-time voice or text conversations with AI characters.
- Serve as a modular project for learning AI engineering, making it easy to replace and experiment with different components.
[Main Capabilities]
- Real-time conversation: Supports real-time talking or messaging with AI characters via web, terminal, and mobile.
- Highly customizable: You can customize the AI character's personality, background, and even voice (through ElevenLabs voice cloning).
- Multi-platform support: Web (React JS, Vanilla JS, WebSockets), mobile (Swift, WebSockets), terminal.
- Latest AI technologies: Integrates OpenAI GPT3.5/4, Anthropic Claude 2, Anyscale Llama2, Chroma, Whisper, ElevenLabs, and more.
- Modular: Easily replace different modules to customize the workflow, offering high flexibility.
- Speech-to-text: Supports local WhisperX, local Whisper, OpenAI Whisper API, Google Speech to Text.
- Text-to-speech: Supports ElevenLabs, Edge TTS, Google Text to Speech.
- Voice cloning: Implemented through ElevenLabs.
- Backend and data: FastAPI, SQLite, Docker, LlamaIndex, Chroma, LangChain.
[How to Use] Quick installation via Docker: copy .env.example to .env and fill in the API key (a single ReByte or OpenAI API key is enough to get started), then run docker compose up and open http://localhost:3000 to use it. Developers can also install via Python: clone the repository, install dependencies such as portaudio and ffmpeg, create the SQLite database, run database upgrades, configure .env, start the backend (python cli.py run-uvicorn or uvicorn realtime_ai_character.main:app), and start the frontend (the web client is located at client/next-web, the terminal client is python client/cli.py, and the mobile client requires opening the project in Xcode). Using GPT4 can provide better conversation results, and wearing headphones is recommended to avoid echo. Remote connection to the RealChar server requires configuring SSL.
[Applicable Scenarios]
- Personal entertainment: Chat with characters such as Santa Claus and AI Elon Musk.
- AI learning and experimentation: Learn AI engineering as a modular project.
- Multi-platform AI assistant deployment: Integrate AI characters on web, mobile, or terminal.
[Limitations] Some features (such as lower conversation latency, persistent conversations, session management, support for Agents/GPTs, and additional TTS services) are provided by the ReByte platform. Twilio phone integration currently only supports US phone numbers and ElevenLabs voice characters. If you need to use a login system, you can get Firebase OAuth2 login for free.
[API Configuration] The project supports a variety of optional API key configurations, including ReByte, OpenAI, Anthropic (Claude 2), Anyscale, Google Speech-to-Text, OpenAI Whisper, ElevenLabs, Google Text-to-Speech, etc., which users can fill in as needed. The default text-to-speech uses the free Edge TTS.
[Creating Characters] You can create characters locally (see realtime_ai_character/character_catalog/README.md), or create them through ReByte.ai (see docs/rebyte_agent_clone_instructions.md).
The project welcomes contributions, with contribution guidelines and a Discord community provided.
