GenoTEX

GenoTEX (Genomics Data Automatic Exploration Benchmark) is a benchmark dataset for automated analysis of gene expression data, specifically designed to evaluate the performance of agents based on large language models (LLMs) on real-world gene expression analysis tasks. This benchmark is expert-annotated, and the related paper has been accepted as an Oral presentation at MLCB 2025. Its goal is to identify disease-related genes while taking the influence of other biological factors into consideration.

Purpose: GenoTEX provides annotated code and results for solving a wide range of gene-trait association (GTA) analysis problems, covering dataset selection, preprocessing, and statistical analysis, and following standard computational genomics workflows. The benchmark includes annotations from bioinformatics experts to ensure accuracy and reliability.

Main capabilities and data scale:

• 1,384 GTA analysis questions: including 132 unconditional questions and 1,252 conditional questions.
• 41.5 GB input data: 911 datasets, with an average of 167 samples per dataset, totaling 152,415 samples.
• 237,907 lines of analysis code: carefully curated by bioinformatics experts, with an average of 261 lines per dataset.
• Three evaluation tasks: dataset selection, data preprocessing, and statistical analysis.
• Comprehensive gene features: an average of 18,530 normalized gene features per dataset.
• Significant gene discovery: significant genes identified for each question.

How to use: GenoTEX provides two distribution methods:

1. GitHub repository + cloud storage: code and documentation are hosted on GitHub, and data is provided through separate cloud storage links (Google Drive, Baidu Netdisk). Total data size is 82.0 GB.
2. Complete package on data platforms: packaged versions of code and data are provided on Kaggle and Hugging Face.

Usage steps:

• Clone the GitHub repository:git clone https://github.com/Liu-Hy/GenoTEX.git, then cd GenoTEX.
• Download the data folders ('metadata', 'input', 'output') from cloud storage and place them in the repository root directory.
• For Kaggle users, run the script to recompress the .gz files.
• Create and activate the conda environment.
• Explore the benchmark: preprocessing Jupyter Notebooks are located in the ./code/{trait_name}/ directory, and the statistical analysis scripts are ./code/regress.py. Run the preprocessing notebooks first, then run the regression scripts.

Evaluating custom methods: Ensure that the method output format is consistent with the benchmark, that is, organized in the same structure as the ./output directory.

Applicable scenarios: Suitable for evaluating automated methods for gene expression data analysis in biomedical research, especially for LLM-based agents. It can be used for automated analysis research in clinical applications such as disease marker identification and personalized medicine.

Limitations:

• Data download requires sufficient disk space (82.0 GB).
• Data organization has specific naming conventions (e.g., removing apostrophes, replacing spaces with underscores).
• Use must comply with relevant licensing and citation requirements.