alterlab-biorxiv

bioRxiv Database

Overview

Python tooling over the keyless bioRxiv API for searching and retrieving life-sciences preprints. Searches by keyword, author, date range, and category, returning structured JSON (titles, abstracts, DOIs, authors, versions), and downloads full-text PDFs.

For published, peer-reviewed literature use alterlab-pubmed; for computer-science / physics / math preprints use alterlab-arxiv. bioRxiv covers biology subjects only. The same API also serves medRxiv (swap biorxiv for medrxiv in the endpoint path, e.g. /details/medrxiv/2025-03-21/2025-03-28?category=cardiovascular%20medicine); the bundled script targets bioRxiv, so query medRxiv with curl/requests directly.

How it works (and its limits)

The bioRxiv /details endpoint returns preprints by date range, 30 records per page. It accepts a server-side subject-category filter (?category=cell_biology), but has no keyword or author filter. So this tool:

Paginates the date range (following the cursor until all records are retrieved), passing --category to the server so only that subject is fetched, then
Filters client-side by keyword (substring over title/abstract) and author (substring over the author list).

Implication: a wide date range means many API calls and a large download. Keep ranges as tight as the question allows, and use --category (cuts API calls) and --limit to bound the work.

When to Use This Skill

Use this skill when:

Searching for recent life-sciences preprints in specific research areas
Tracking preprints by particular authors
Conducting systematic preprint literature reviews
Analyzing preprint trends over time periods
Retrieving metadata for citation management
Downloading preprint PDFs for analysis
Filtering papers by bioRxiv subject categories

Does NOT Trigger

Scenario	Use Instead
Peer-reviewed, MeSH-indexed biomedical journal articles	`alterlab-pubmed`
CS / physics / math / quantitative-biology preprints on arXiv	`alterlab-arxiv`
Citation counts or cross-publisher bibliometrics for preprints	`alterlab-openalex`
Depositing your own preprint (server choice, license, versioning)	`alterlab-preprint-deposition`
Multi-database systematic review with PRISMA screening	`alterlab-literature-review`

Running the script

The script's only dependency is requests. Run it with uv so the dependency is provisioned on the fly:

uv run --with requests scripts/biorxiv_search.py --help

The python scripts/biorxiv_search.py ... invocations below are shorthand; substitute uv run --with requests scripts/biorxiv_search.py ... (or activate an environment that has requests).

Core Search Capabilities

1. Keyword Search

Search for preprints containing specific keywords in titles, abstracts, or author lists.

Basic Usage:

python scripts/biorxiv_search.py \
  --keywords "CRISPR" "gene editing" \
  --start-date 2024-01-01 \
  --end-date 2024-12-31 \
  --output results.json

With Category Filter:

python scripts/biorxiv_search.py \
  --keywords "neural networks" "deep learning" \
  --days-back 180 \
  --category neuroscience \
  --output recent_neuroscience.json

Search Fields: Keyword matching is a case-insensitive substring match, and a paper matches if any keyword is found (OR semantics, not AND). By default keywords are searched in both title and abstract. Customize with --search-fields:

python scripts/biorxiv_search.py \
  --keywords "AlphaFold" \
  --search-fields title \
  --days-back 365

2. Author Search

Find all papers by a specific author within a date range.

Basic Usage:

python scripts/biorxiv_search.py \
  --author "Smith" \
  --start-date 2023-01-01 \
  --end-date 2024-12-31 \
  --output smith_papers.json

Recent Publications:

# Last year by default if no dates specified
python scripts/biorxiv_search.py \
  --author "Johnson" \
  --output johnson_recent.json

3. Date Range Search

Retrieve all preprints posted within a specific date range.

Basic Usage:

python scripts/biorxiv_search.py \
  --start-date 2024-01-01 \
  --end-date 2024-01-31 \
  --output january_2024.json

With Category Filter:

python scripts/biorxiv_search.py \
  --start-date 2024-06-01 \
  --end-date 2024-06-30 \
  --category genomics \
  --output genomics_june.json

Days Back Shortcut:

# Last 30 days
python scripts/biorxiv_search.py \
  --days-back 30 \
  --output last_month.json

4. Paper Details by DOI

Retrieve detailed metadata for a specific preprint. bioRxiv DOIs come in two prefixes: 10.1101/… for older preprints and 10.64898/… for preprints posted since the move to openRxiv (December 2025), e.g. 10.64898/2026.08.28.747819. Both work with /details/ and the www.biorxiv.org/content/ URLs; the script normalizes DOIs, doi.org links, and content URLs regardless of prefix. Don't hard-code 10.1101 in regexes or validators.

Basic Usage:

python scripts/biorxiv_search.py \
  --doi "10.1101/2024.01.15.123456" \
  --output paper_details.json

Full DOI URLs Accepted (either prefix):

python scripts/biorxiv_search.py \
  --doi "https://doi.org/10.64898/2026.08.28.747819"

5. PDF Downloads

Download the full-text PDF of any preprint.

Basic Usage:

python scripts/biorxiv_search.py \
  --doi "10.1101/2024.01.15.123456" \
  --download-pdf paper.pdf

Batch Processing: For multiple PDFs, extract DOIs from a search result JSON and download each paper:

import json
from biorxiv_search import BioRxivSearcher

# Load search results
with open('results.json') as f:
    data = json.load(f)

searcher = BioRxivSearcher(verbose=True)

# Download each paper
for i, paper in enumerate(data['results'][:10]):  # First 10 papers
    doi = paper['doi']
    searcher.download_pdf(doi, f"papers/paper_{i+1}.pdf")

Valid Categories

Filter searches by bioRxiv subject categories:

animal-behavior-and-cognition
biochemistry
bioengineering
bioinformatics
biophysics
cancer-biology
cell-biology
clinical-trials
developmental-biology
ecology
epidemiology
evolutionary-biology
genetics
genomics
immunology
microbiology
molecular-biology
neuroscience
paleontology
pathology
pharmacology-and-toxicology
physiology
plant-biology
scientific-communication-and-education
synthetic-biology
systems-biology
zoology

Output Format

All searches return structured JSON with the following format:

{
  "query": {
    "keywords": ["CRISPR"],
    "start_date": "2024-01-01",
    "end_date": "2024-12-31",
    "category": "genomics"
  },
  "result_count": 42,
  "results": [
    {
      "doi": "10.1101/2024.01.15.123456",
      "title": "Paper Title Here",
      "authors": "Smith, J.; Doe, J.; Johnson, A.",
      "author_corresponding": "Smith J",
      "author_corresponding_institution": "University Example",
      "date": "2024-01-15",
      "version": "1",
      "type": "new results",
      "license": "cc_by",
      "category": "genomics",
      "abstract": "Full abstract text...",
      "pdf_url": "https://www.biorxiv.org/content/10.1101/2024.01.15.123456v1.full.pdf",
      "html_url": "https://www.biorxiv.org/content/10.1101/2024.01.15.123456v1",
      "jatsxml": "https://www.biorxiv.org/content/...",
      "published": ""
    }
  ]
}

Common Usage Patterns

Literature Review Workflow

Broad keyword search:

python scripts/biorxiv_search.py \
  --keywords "organoids" "tissue engineering" \
  --start-date 2023-01-01 \
  --end-date 2024-12-31 \
  --category bioengineering \
  --output organoid_papers.json

Extract and review results:

import json

with open('organoid_papers.json') as f:
    data = json.load(f)

print(f"Found {data['result_count']} papers")

for paper in data['results'][:5]:
    print(f"\nTitle: {paper['title']}")
    print(f"Authors: {paper['authors']}")
    print(f"Date: {paper['date']}")
    print(f"DOI: {paper['doi']}")

Download selected papers:

from biorxiv_search import BioRxivSearcher

searcher = BioRxivSearcher()
selected_dois = ["10.1101/2024.01.15.123456", "10.1101/2024.02.20.789012"]

for doi in selected_dois:
    filename = doi.replace("/", "_").replace(".", "_") + ".pdf"
    searcher.download_pdf(doi, f"papers/{filename}")

Trend Analysis

Track research trends by analyzing publication frequencies over time:

python scripts/biorxiv_search.py \
  --keywords "machine learning" \
  --start-date 2020-01-01 \
  --end-date 2024-12-31 \
  --category bioinformatics \
  --output ml_trends.json

Then analyze the temporal distribution in the results.

Author Tracking

Monitor specific researchers' preprints:

# Track multiple authors (each run scans the whole window, so keep it short)
for author in Smith Johnson Williams; do
  python scripts/biorxiv_search.py \
    --author "$author" \
    --days-back 365 \
    --output "${author}_papers.json"
done

Python API Usage

For more complex workflows, import and use the BioRxivSearcher class directly:

from scripts.biorxiv_search import BioRxivSearcher

# Initialize
searcher = BioRxivSearcher(verbose=True)

# Multiple search operations
keywords_papers = searcher.search_by_keywords(
    keywords=["CRISPR", "gene editing"],
    start_date="2024-01-01",
    end_date="2024-12-31",
    category="genomics"
)

author_papers = searcher.search_by_author(
    author_name="Smith",
    start_date="2023-01-01",
    end_date="2024-12-31"
)

# Get specific paper details
paper = searcher.get_paper_details("10.1101/2024.01.15.123456")

# Download PDF
success = searcher.download_pdf(
    doi="10.1101/2024.01.15.123456",
    output_path="paper.pdf"
)

# Format results consistently
formatted = searcher.format_result(paper, include_abstract=True)

Best Practices

Keep date ranges tight: Because filtering is client-side, the tool paginates the entire range (30 records/page) before filtering. A single busy week is ~800 preprints (~27 API calls); a full year is tens of thousands. Narrow the range, or use --days-back for recency.
Filter by category: Use --category whenever the subject is known. It is sent to the server (?category=), so it cuts both the result set and the number of API calls (one busy week: ~1,270 preprints overall vs. ~80 in cell biology).
Cap with --limit: For pure date-range searches, --limit also stops pagination early, so it genuinely reduces API calls. For keyword/author searches the whole range must be scanned first, so --limit only trims the final list.
Respect rate limits: The script sleeps 0.5s between requests. There is no documented hard rate limit, but for large collections add more delay and cache results to JSON.
Version tracking: Preprints can have multiple versions. DOI lookups return the latest version; download_pdf resolves the latest version automatically (pass version= to override). PDF/HTML URLs embed the version number. The published field of /details/ carries the journal DOI once the preprint is published (or NA) — use it rather than the per-DOI /pubs/ lookup, which returns nothing for 10.64898 DOIs.
PDF downloads can be throttled: PDFs come from www.biorxiv.org, which sits behind Cloudflare and may answer scripted requests with HTTP 429. Space downloads out, fall back to the html_url, and for bulk full text use bioRxiv's requester-pays text-mining bucket s3://biorxiv-src-monthly (MECA zip packages; see https://www.biorxiv.org/tdm).
Handle empty results: Check result_count. Empty results usually mean the date range had no matching papers, an over-narrow category, or transient API connectivity issues — not a silent truncation (pagination retrieves the full range).
Verbose mode for debugging: Use --verbose to see each paginated API request and the reported total.

Advanced Features

Custom Date Range Logic

from datetime import datetime, timedelta
from scripts.biorxiv_search import BioRxivSearcher

# Last quarter
end_date = datetime.now()
start_date = end_date - timedelta(days=90)

papers = BioRxivSearcher().search_by_date_range(
    start_date.strftime("%Y-%m-%d"), end_date.strftime("%Y-%m-%d"), category="genomics"
)

Result Limiting

Limit the number of results returned:

python scripts/biorxiv_search.py \
  --keywords "COVID-19" \
  --days-back 30 \
  --limit 50 \
  --output covid_top50.json

Exclude Abstracts for Speed

When only metadata is needed:

# Note: Abstract inclusion is controlled in Python API
from scripts.biorxiv_search import BioRxivSearcher

searcher = BioRxivSearcher()
papers = searcher.search_by_keywords(keywords=["AI"], days_back=30)
formatted = [searcher.format_result(p, include_abstract=False) for p in papers]

Programmatic Integration

Integrate search results into downstream analysis pipelines:

import json
import pandas as pd

# Load results
with open('results.json') as f:
    data = json.load(f)

# Convert to DataFrame for analysis
df = pd.DataFrame(data['results'])

# Analyze
print(f"Total papers: {len(df)}")
print(f"Date range: {df['date'].min()} to {df['date'].max()}")
print(f"\nTop authors by paper count:")
print(df['authors'].str.split(';').explode().str.strip().value_counts().head(10))

# Filter and export
recent = df[df['date'] >= '2024-06-01']
recent.to_csv('recent_papers.csv', index=False)

Reference Documentation

For detailed API specifications, endpoint documentation, and response schemas, refer to:

references/api_reference.md - Complete bioRxiv API documentation

The reference file includes:

Full API endpoint specifications
Response format details
Error handling patterns
Rate limiting guidelines
Advanced search patterns

About this skill

Overview

How it works (and its limits)

When to Use This Skill

Does NOT Trigger

Running the script

Core Search Capabilities

1. Keyword Search

2. Author Search

3. Date Range Search

4. Paper Details by DOI

5. PDF Downloads

Valid Categories

Output Format

Common Usage Patterns

Literature Review Workflow

Trend Analysis

Author Tracking

Python API Usage

Best Practices

Advanced Features

Custom Date Range Logic

Result Limiting

Exclude Abstracts for Speed

Programmatic Integration

Reference Documentation