What is OmniSQL
OmniSQL is an open-source text-to-SQL model that efficiently converts natural language questions into SQL query statements. Through an innovative data synthesis framework, it generated the first million-scale text-to-SQL dataset, SynSQL-2.5M, containing 2.5 million high-quality samples, covering more than 16,000 cross-domain databases, with samples spanning multiple complexity levels and language styles. OmniSQL provides three model versions: 7B, 14B, and 32B, and during fine-tuning it incorporates high-quality annotated data from Spider and BIRD.
Main features of OmniSQL
- Text-to-SQL conversion: OmniSQL can understand questions users pose in natural language and convert them into corresponding SQL query statements.
- Support for multiple databases and complex queries: OmniSQL supports multiple database types and can handle SQL queries of various complexity levels, from simple single-table queries to complex multi-table joins, subqueries, function calls, and common table expressions (CTEs).
- Provides chain-of-thought solutions: In addition to generating SQL query statements, OmniSQL provides a chain-of-thought solution for each sample. This chain of thought shows the logical reasoning process from understanding the natural language question to generating the SQL query, helping users better understand the model's decision path and also making it easier for developers to debug and optimize the model.
- Multiple model version options: OmniSQL provides three model versions of different sizes, namely 7B, 14B, and 32B. Users can choose the appropriate model version based on their actual needs and computing resources. Models of different scales strike a balance between performance and resource consumption. Smaller models run faster and use fewer resources, while larger models may perform better in certain complex query scenarios.
Technical principles of OmniSQL
- Automatic database generation: OmniSQL analyzes web tables, infers business scenarios, and with the help of large language models automatically constructs database structures containing multi-table relationships and primary/foreign key constraints. It adopts enhancement strategies to increase the number of columns and optimize the structure, making the generated databases more aligned with real-world applications.
- Complexity-aware SQL query generation: It defines four complexity levels and, combined with the SQLite function library, such as aggregate functions (SUM, AVG, etc.) and window functions (ROW_NUMBER, RANK, etc.), generates various types of SQL queries. It can intelligently select the complexity level based on the user's question and provide an appropriate query statement.
- Stylized question back-translation: It adopts a SQL-to-Question strategy, translating SQL queries back into natural language questions in 9 language styles. Semantic analysis ensures that the meaning remains consistent before and after translation, improving the efficiency and accuracy of natural language and SQL conversion and adapting to different users' language habits.
- CoT solution synthesis: Through a step-by-step reasoning generator, intermediate derivation steps are added to samples. During training, the model learns the conversion from questions to SQL and also learns the reasoning logic of each step, improving reasoning accuracy and reliability, showing users a transparent reasoning process, and enhancing trust.
- Large-scale data synthesis and training: OmniSQL, based on its data synthesis framework, generated a large-scale high-quality training dataset, SynSQL-2.5M. The dataset contains more than 2.5 million samples and covers more than 16,000 cross-domain databases. By training on such a large-scale and diverse dataset, OmniSQL can learn the mapping relationships between natural language expressions and SQL queries across different domains and styles, giving it stronger generalization ability and adaptability.
OmniSQL's project address
- Github repository：https://github.com/RUCKBReasoning/OmniSQL
- arXiv technical paper：https://arxiv.org/pdf/2503.02240
OmniSQL's application scenarios
- Enterprise data analysis: OmniSQL's natural language query feature allows non-technical personnel to easily obtain the information they need from databases.
- Education: In SQL teaching, OmniSQL's chain-of-thought (CoT) solution can help beginners better understand the process of converting natural language questions into SQL queries. Teachers can use OmniSQL to generate query examples, allowing students to master SQL concepts and techniques through hands-on practice.
- Cross-domain adaptation: Based on its data synthesis framework, OmniSQL can quickly generate datasets for specific domains. In the healthcare field, it can generate the EHRSQL dataset to support medical research; in the scientific research field, it can generate the ScienceBenchmark dataset to assist scientific data analysis.