The failure this skill prevents: a page states a default, a precedence order, or a field name that reads plausibly, passes Vale and the link checker, and is wrong. Nothing in CI checks whether a sentence about product behavior is true. A reviewer who knows the product catches some of it; the rest reaches readers.
Published docs, a README, and a model's recollection are all secondary sources. Prefer evidence you produced in this session.
Step 1. Pick the strongest evidence available
Work down this list and stop at the first rung you can actually reach. Each rung is weaker than the one above it.
- Run the sample.
make test-code-samples, or scope it withFILES="src/code-samples/langchain/foo.py". This is the only method that proves a sample works end to end. It needs provider API keys in the environment, so it often fails locally and runs in CI instead. - Run the half that needs no model. Most claims are about a library's behavior, not the model's. Stand up the real object in a scratch script and print what it returns. A local MCP server, an adapter, a parser, or a store needs no API key, and the answer it gives is exact rather than inferred.
- Read the library source.
gh api repos/langchain-ai/<repo>/contents/<path> --jq '.content' | base64 -d, orgh api "search/code?q=<symbol>+repo:langchain-ai/<repo>"to locate the file first. Settles what a type is, what a default is, and which branch actually runs. - Check the installed package.
uv run python -c "from x import Y"confirms an import path resolves at the version this repo pins, which source onmaincannot tell you. - Look up the signature. The
reference-langchainMCP server exposessearch_apiandget_symbolover reference.langchain.com. Right for "what are the parameters of X", not for "what does X do at runtime".
Step 2. Know which repository owns the claim
| Product | Repository | Where to look |
|---|---|---|
| LangChain | langchain-ai/langchain | libs/langchain_v1/langchain/ for v1, libs/core/ for core, libs/partners/ for provider packages |
| LangGraph | langchain-ai/langgraph | libs/langgraph/, libs/prebuilt/, libs/checkpoint*/, libs/cli/, libs/sdk-py/ and libs/sdk-js/ |
| Deep Agents | langchain-ai/deepagents | Also the source of the eval matrix the docs publish |
| LangSmith SDK | langchain-ai/langsmith-python and langchain-ai/langsmith-javascript | The Python and JavaScript client libraries |
| LangSmith platform | langchain-ai/langchainplus (private) | smith-backend/ (Python API), smith-go/ (Go services), smith-frontend/ (UI labels and flows), host-backend/, lc_config/ for settings and their defaults |
| Agent Server | langchain-ai/langgraph-api (private) | Not in the OSS langgraph repo. Also the source of the Agent Server OpenAPI spec PRs |
| Helm charts | langchain-ai/helm | charts/langsmith/values.yaml for defaults, templates/_helpers.tpl for the value-to-environment-variable mapping |
| OpenEvals | langchain-ai/openevals | Prebuilt evaluators |
Two traps worth naming:
- A private repo is still readable through
gh.gh apiworks on langchainplus and langgraph-api with the usual credentials. GitHub code search is unreliable on the larger ones, so fall back to the git trees API and fetch a path directly. - langchain-ai/deployments is archived. Its GitOps content moved into langchainplus. A skill, script, or note that still points at it is stale.
A self-hosted LangSmith claim usually spans two repositories: the chart sets a value, and the backend decides what to do with it. Check both before describing precedence, because the chart alone does not tell you which setting wins.
Step 3. Verify the claim, not the sentence
Read the code path that produces the behavior, rather than a name that sounds like it. Three failures that recur:
- A resolution order inferred from a settings table. Find the query or the
conditional that picks a value. An
ORDER BY, aCOALESCE, or anorexpression is the precedence; a list of configurable fields is not. - A defensive pattern copied from another sample. Check whether the guarded
case can occur. If the constructor only builds an object when a field is
present, a
.getwith a default is documenting a state that never exists. - A version or model identifier from memory. Fetch it. The repository's own
internal consensus is evidence about the repository, not about the provider.
For model IDs specifically, see the Model references section of
AGENTS.md.
Step 4. Record what you checked, and what you could not
Name the file and symbol in the pull request body, not just the conclusion. A reviewer can then disagree with the evidence rather than with an assertion.
State the gaps in the same place. Common ones that no amount of source reading closes:
- A UI string. Source gives you a field and a component; only the running product gives you the rendered label. Describe the behavior rather than quoting a label you have not seen.
- The model-dependent half of a sample. Say that CI has yet to run it.
- A beta surface.
langchain.mcpand similar namespaces can move under a page that was accurate when written.
Writing "not verified: the visible label of this control" is a better outcome than a confident sentence a reader discovers is wrong.
Related
AGENTS.md holds the rules that apply to every edit, including never
fabricating an example and testing code before publishing it. Use
docs-code-samples when the claim belongs in a runnable sample rather than in
prose, and docs-review for the style pass once the facts are settled.