sn-search-academic

sn-search-academic - Academic Search

Credential Configuration

It is recommended to write API keys, tokens, and cookies uniformly in the repository root directory .env(refer to .env.example), and have the runtime or user load them as environment variables with the same names before execution. Scripts still only read credentials from environment variables or explicit CLI parameters; do not write real keys into skill payloads, reports, logs, or commits.

Use three unified entry points to complete academic research:

search.py: search papers and encyclopedia entries
paper.py: list paper sections, read the full text of a paper or a specified section
refTree.py: query a paper's references and citations

Do not directly call historical provider scripts; they are merely internal implementation details of the unified entry points. When you need the provider fallback chain, parameter dispatch, or complete output fields, read as needed references/search.md、references/paper.md、references/refTree.md。

Available Scripts

Script	Purpose	Main Input	Main Output
`scripts/search.py`	Search papers/encyclopedia	`query`, optional `--source`、`--limit`、`--category`、`--lang`	Paper/encyclopedia entries grouped by source, located in `source_results[*].items`
`scripts/paper.py`	List sections, read the full text of a paper or a section	Paper ID, optional `--source`、`--list_section`、`--section`	Section list located in `sections`; full text or section body located in `content`
`scripts/refTree.py`	Query citation tree	`--paper_id`、`--title`, optional `--direction`	References and citing papers, located in `source_results[].references` / `source_results[].citations`

Execution Conventions

This skill's scripts/...、requirements.txt、references/... paths are all relative to this skill directory; if the current working directory is different, resolve them to absolute paths first, and do not rely on ${SKILL_DIR} runtime variables.

Invocation conventions:

Do not start multiple scripts of this skill in parallel;search.py and refTree.py already handle concurrency, timeouts, and the provider fallback chain internally.
For long results, prefer adding --output <path> to write to a file, then read the necessary fields, to avoid excessively long terminal output.
--provider-timeout indicates a single provider timeout; by default, use the script's built-in timeout.

Dependencies

On first run or when a script prompts that a library is missing, use this skill's dependency manifest to install into the current Python environment:

python3 -m pip install -r requirements.txt

Do not automatically install dependencies inside scripts. If installation fails, the network is unavailable, or a package is unavailable, stop using the corresponding script and switch to WebSearch/browser-use, explaining that dependencies are missing.

Crawler fallback also requires an additional runtime environment:

python3 -m playwright install firefox

arxiv_crawler_search.py and semantic_scholar_crawler_refTree.py also require Node.js, as well as a camoufox-js with node_modulesinstalled in the current directory or an ancestor directory. When these environments are missing, do not try to bypass them; switch to a non-crawler provider or web search.

Parameter Description

search.py

Unified search entry point. By default, searches all supported sources and returns results grouped by source.

python3 scripts/search.py <query> [选项]

Parameter	Description	Default Value
`query`	Search keywords, required positional parameter	-
`--source`, `--sources`, `-s`	Search source; supports repeated parameters or comma-separated values	`all`
`--limit`, `-n`	Number of results returned per source	`10`
`--category`, `-c`	ArXiv category filter, only passed to sources that support categories	-
`--lang`, `-l`	Language hint, only passed to sources that support the language parameter	-
`--output`, `-o`	Write the final JSON to a file	-
`--provider-timeout`	Timeout per provider, in seconds;`0` means no limit	`60`

Supported --source：

all
arxiv
semantic
google_scholar
pubmed
ssrn
wikipedia

Example:

python3 scripts/search.py "retrieval augmented generation" --limit 5
python3 scripts/search.py "diffusion model" --source arxiv,semantic --category cs.CV --limit 5
python3 scripts/search.py "阿尔茨海默病 多模态诊断" --source pubmed,wikipedia --lang zh --limit 5
python3 scripts/search.py "open source governance" --source ssrn --limit 10
python3 scripts/search.py "agentic memory" --source all --limit 8 --output results/search.json

paper.py

Unified paper reading entry point. Reads arXiv papers by default; when reading PMC papers, explicitly pass --source pmc. When unsure of the section name, first use --list_section to list available sections, then use --section for close reading.

python3 scripts/paper.py <id> [选项]

Parameter	Description	Default value
`id`	Paper ID. arXiv supports raw IDs,`arXiv:` prefix, abs/pdf URLs; PMC supports `PMC11119143`、`11119143`、PMC URL	-
`--source`	Paper source:`arxiv` or `pmc`	`arxiv`
`--section`, `-s`	Read the specified section; if omitted, read the full text	-
`--list_section`, `--list-section`	List the paper's available sections without returning the body text; cannot be used together with `--section` at the same time	false
`--output`, `-o`	Write the final JSON to a file	-

Example:

python3 scripts/paper.py 2603.00729
python3 scripts/paper.py 2603.00729 --list_section
python3 scripts/paper.py arXiv:2603.00729 --section introduction
python3 scripts/paper.py 2603.00729 --section method --output results/paper-method.json
python3 scripts/paper.py PMC11119143 --source pmc
python3 scripts/paper.py PMC11119143 --source pmc --list-section
python3 scripts/paper.py PMC11119143 --source pmc --section results

refTree.py

Unified citation tree entry point.--paper_id and --title are both required; the title is used for exact matching during fallback.

python3 scripts/refTree.py --paper_id <paper_id> --title <title> [选项]

Parameter	Description	Default value
`--paper_id`	Paper ID: Semantic Scholar ID, DOI, ArXiv ID, PMID, etc.	-
`--title`	Paper title, required	-
`--direction`	Query direction:`references` or `citations`; if omitted, both are queried	-
`--source`, `--sources`, `-s`	Citation tree source; currently supports `all`、`semantic`	`all`
`--limit`, `-n`	Number of results returned per source, per direction	`10`
`--api-key`	Semantic Scholar API key, optional	-
`--provider-timeout`	Timeout per provider, in seconds;`0` means no limit	`60`
`--output`, `-o`	Write the final JSON to a file	-

Note: the parameter name is --paper_id, not --paper-id；paper_id does not support positional arguments.

Example:

python3 scripts/refTree.py --paper_id "2309.16609" --title "Qwen Technical Report"
python3 scripts/refTree.py --paper_id "2309.16609" --title "Qwen Technical Report" --direction references --limit 20
python3 scripts/refTree.py --paper_id "10.1038/s41586-024-07487-w" --title "AlphaFold 3" --direction citations
python3 scripts/refTree.py --paper_id "2309.16609" --title "Qwen Technical Report" --output results/refTree.json

Output format

All scripts output JSON. First check the top-level success; on failure, read error、errors and attempts to determine whether it is no results, timeout, or provider failure.

search.py output

The CLI output's top level does not contain items, the paper entries are in source_results[*].items :

{
  "success": true,
  "query": "retrieval augmented generation",
  "provider": "search.py",
  "sources": ["arxiv", "semantic"],
  "source_results": [
    {
      "source": "arxiv",
      "success": true,
      "provider": "arxiv_official",
      "items": [
        {
          "source": "arxiv",
          "provider": "arxiv_official",
          "title": "Example title",
          "abstract": "Example abstract",
          "citation_count": null,
          "arxiv_id": "2301.00001",
          "url": "https://arxiv.org/abs/2301.00001"
        }
      ],
      "attempts": [],
      "error": null
    }
  ],
  "errors": [],
  "error": null
}

Common item fields:

General:title、abstract、snippet、url、citation_count、doi
arXiv：arxiv_id、pdf_url、categories
Semantic Scholar：paper_id、venue、year
SSRN：doi、year、publication_date、publisher、container（container is the SSRN working paper series name, e.g. "SSRN Electronic Journal")
PubMed：pmid、pmc_id、journal、pub_date
Wikipedia：page_id、word_count、section_title

paper.py output

Reads the full text by default; when --section is specified, reads a section. The body text is in the top-level content：

{
  "success": true,
  "source": "arxiv",
  "provider": "arxiv_html",
  "arxiv_id": "2603.00729",
  "section": "introduction",
  "content": "<全文或章节正文>",
  "char_count": 12345,
  "attempts": [],
  "error": null
}

When --list_section is specified, only the section structure is returned, not content：

{
  "success": true,
  "source": "arxiv",
  "provider": "arxiv_html",
  "arxiv_id": "2603.00729",
  "section_count": 2,
  "sections": [
    {"name": "Abstract", "level": 0},
    {"name": "1 Introduction", "level": 1}
  ],
  "attempts": [],
  "error": null
}

Common fields:

arXiv：arxiv_id、title、abs_url、html_url、pdf_url、section_count、sections
PMC：pmc_id、pmid、title、pmc_url、section_count、sections
When --list_section is specified, returns sections and section_count, excluding content
When --section is specified, it includes section; when --list_section / --section is not specified, reads the full text

refTree.py output

Citation tree results are in source_results[*].references and source_results[*].citations：

{
  "success": true,
  "id": "2309.16609",
  "title": "Qwen Technical Report",
  "provider": "refTree.py",
  "direction": "all",
  "source_results": [
    {
      "source": "semantic",
      "success": true,
      "provider": "semantic_official",
      "references": [
        {
          "title": "Example reference",
          "abstract": "Example abstract",
          "citation_count": 128,
          "paper_id": "example-reference-id",
          "arxiv_id": "2301.00001"
        }
      ],
      "citations": [
        {
          "title": "Example citing paper",
          "abstract": "Example abstract",
          "citation_count": 42,
          "paper_id": "example-citing-id",
          "doi": "10.1234/example"
        }
      ],
      "attempts": [],
      "error": null
    }
  ],
  "errors": [],
  "error": null
}

If using --output, all three scripts will additionally include in the JSON output_path。

Concurrency and rate limiting conventions

These scripts access external academic services and must control request frequency. When executing this skill's scripts:

Do not run multiple search scripts concurrently.
Do not use parallel tools to call multiple python3 scripts/... commands at the same time.
Run only one script command at a time, and wait for the result to return before running the next one.
For batch queries, prefer using the script's built-in --limit、--id-list and other parameters instead of starting multiple processes.
If continuous calls are needed, execute them in order and wait a few seconds when necessary.

Full-text reading workflow

When search results only have abstracts, use paper.py to list sections first, then supplement with the full text or key sections.

First use search.py to search, and prioritize recording from source_results[*].items the title、arxiv_id、pmc_id、paper_id、doi、citation_count。
If an entry has arxiv_id, first use python3 scripts/paper.py <arxiv_id> --source arxiv --list_section to view sections; then use --section <section> for close reading.
If an entry has pmc_id, first use python3 scripts/paper.py <pmc_id> --source pmc --list_section to view sections; then read --section <section> 。
If an overall understanding is needed, then read the full text without --section / --list_section .
When the full text is very long, use --output results/paper.json, then read fields such as content、sections、char_count .

Citation tracing workflow

Discover related work not covered by keyword search through a paper's citation relationships.

Use references to find foundational work, and citations to find subsequent progress.refTree.py requires passing both the paper ID and title.

Backward tracing (finding foundational work)：

Keyword search finds highly relevant papers → take their paper_id or arxiv_id and title
refTree.py --paper_id "<id>" --title "<title>" --direction references --limit 20 → find highly cited references
Filter entries relevant to the research question → use paper.pyfor in-depth reading

Forward tracking (finding subsequent progress)：

Find the field's foundational paper or key paper → take its ID
refTree.py --paper_id "<id>" --title "<title>" --direction citations --limit 20 → find recent highly cited follow-up work
Filter entries relevant to the research question → use paper.pyfor in-depth reading

Citation chain: building an evolution path

Starting from seed paper A → backward find A's key reference B
Starting from B → forward find subsequent work citing B (possibly discovering related paper C not cited by A)
Form the knowledge脉络 of B → A → ... and B → C → ...

Main workflow

Strictly follow this workflow to execute the entire academic search process

Select all possible platforms from the provided academic platforms to search academic literature
If the abstract is insufficient or the paper is highly relevant, list the sections, try to read the paper's sections or full text, and judge the relevance of the paper to the search requirement
Select highly relevant papers and search their references and citations (using the citation tracing workflow).
Select highly cited literature in the citation tree and execute steps 2 and 3.
Repeat the above steps, conduct multiple rounds of searching, and search as much as possible.
Stop searching when the number of documents, citation chains, and full-text evidence are sufficient to support the answer, and explain the main basis in the conclusion.

ArXiv classification quick reference

Top-level fields can be used directly (such as --category cs), subcategories are more precise (such as --category cs.AI）。

Field	Classification code	Description
Computer Science	`cs.AI`	Artificial Intelligence
	`cs.LG`	Machine Learning
	`cs.CL`	Computational Linguistics / NLP
	`cs.CV`	Computer Vision
	`cs.IR`	Information Retrieval
	`cs.RO`	Robotics
	`cs.SE`	Software Engineering
	`cs.DC`	Distributed/Parallel Computing
	`cs.NI`	Networking and the Internet
	`cs.CR`	Cryptography and Security
	`cs.DB`	Databases
	`cs.HC`	Human-Computer Interaction
Statistics	`stat.ML`	Statistical Machine Learning
	`stat.AP`	Applied Statistics
	`stat.ME`	Statistical Methodology
Mathematics	`math.OC`	Optimization and Control
	`math.ST`	Statistical Theory
	`math.CO`	Combinatorics
Physics	`physics`	Physics (All Categories)
	`cond-mat`	Condensed Matter Physics
	`quant-ph`	Quantum Physics
	`hep-th`	High-Energy Theoretical Physics
Economics/Finance	`econ.GN`	General Economics
	`q-fin.CP`	Computational Finance
	`q-fin.ST`	Statistical Finance
Biology/Medicine	`q-bio.NC`	Neuroscience
	`q-bio.GN`	Genomics
	`q-bio.QM`	Quantitative Methods

About this skill

Credential Configuration

Available Scripts

Execution Conventions

Dependencies

Parameter Description

search.py

paper.py

refTree.py

Output format

search.py output

paper.py output

refTree.py output

Concurrency and rate limiting conventions

Full-text reading workflow

Citation tracing workflow

Main workflow

ArXiv classification quick reference