The APIs are not the apps. ChatGPT, Gemini, Perplexity and Google AI Mode add their own search, memory, location and system prompts. This skill asks the same questions in the apps through the user's own Chrome, records each answer with answerrank record, and scores everything with answerrank report.
Before you start (ask, don't assume)
-
Run directory. It needs
queries.jsonandprofile.json. If there isn't one, run theanswerrankskill up toanswerrank run --stop-after queriesfirst. -
Browser. Use a tool that drives the user's real Chrome, where they are already signed in, not a fresh headless browser:
- Claude Code: the Claude in Chrome extension.
- Codex or another agent: a Chrome DevTools or Playwright MCP attached to the user's running Chrome.
If no such tool is connected, stop and say what's missing.
-
Get the user's go-ahead, and tell them:
- which sites, and how many questions per site (default: 10, a mix of intents);
- that it uses their signed-in accounts;
- that answers depend on their account, memory, location and plan;
- that the sites' terms may restrict automated use, so this is meant for small spot checks they watch, not bulk collection.
Surfaces
Label for --model |
Open | Clean-slate mode | Answer is done when | Answer text |
|---|---|---|---|---|
chatgpt.com |
https://chatgpt.com/?temporary-chat=true | That URL starts a temporary chat. It can still use memory and custom instructions, so say so in --note |
"ChatGPT can make mistakes" appears below the answer | main text between "ChatGPT said:" and that footer |
claude.ai |
https://claude.ai/new?incognito | That URL starts an incognito chat: no history or memory | the last [data-is-streaming] element is "false" |
that element's text |
gemini.google.com |
https://gemini.google.com/app | Click "Turn on temporary chat" (top right) for every new chat, then check the page says "Just stopping by?" | no "Stop" button | the last message-content element |
perplexity.ai |
https://www.perplexity.ai | None; note in --note whether you're signed in |
the answer has stopped growing | the last .prose element |
google-ai-mode |
https://www.google.com/search?udm=50&q=QUESTION | None; depends on the Google account | the text has stopped changing (~20 s) | page text after the question, up to "Ask anything" or "Show all" |
google-ai-overview |
https://www.google.com/search?q=QUESTION | None | page loaded | the AI Overview block; record only when one appears |
These selectors were checked in October 2026, and the apps change often. If one stops matching, use the page's visible text and cut out the answer around the question.
For each question, on each surface
- Open a new chat or search, then paste the question exactly as written in
answerrank questions <run_dir>. - Wait until the answer has finished: the stop button is gone and the text has stopped changing for a few seconds.
- Copy the full answer text verbatim, never summarized, plus the source links the answer shows (
a[href^="http"]inside the answer or the sources panel). Skip ads, and skip links back to the app itself.- Long answers are cheaper to move through the clipboard than through your context. With JavaScript, put the answer text into a temporary
<textarea>, focus it and select it. Then press Cmd/Ctrl+C as a real key press: scriptedclipboard.writeTextandexecCommand('copy')are blocked without a user gesture. Remove the textarea, then read the clipboard into a file. On macOS:LANG=en_US.UTF-8 pbpaste > /tmp/answer.txt; without the locale the text comes out mangled. On Linux usexclip -o, on WindowsGet-Clipboard. - Don't return the page URL or query string from JavaScript; some browser tools block results that contain them.
- Long answers are cheaper to move through the clipboard than through your context. With JavaScript, put the answer text into a temporary
- Record the answer from that file:
answerrank record <run_dir> --model chatgpt.com --qid q007 --file /tmp/answer.txt \ --source https://example.com/a --source https://example.com/b \ --note "signed in, temporary chat, US" - Wait 20–60 seconds before the next question, and do at most about 20 questions per site per session.
Stop and hand back to the user when
- a CAPTCHA, "verify you are human", rate-limit or sign-in page appears. Never try to solve or bypass it;
- a site shows a usage limit or warning;
- anything asks to change account settings, accept terms, or pay.
Never type passwords. Never change settings beyond starting a new chat, and never click links inside the answers.
Finish
answerrank report <run_dir> --json
report judges the new answers with the run's judge model, so it needs that model's key. Then compare each surface's score with the API models. A large gap means the app's search or personalization changes the picture. Also list which sources each surface cited, and which questions any app answered without naming the product.