scraping-specialist

Scraping Specialist

Provides website scraping, structured data extraction, browser automation, site mapping, search ad intelligence, social media content collection, comment analysis, and vertical site collection capabilities. Use when tasks involve scrape, crawl, extract, web page scraping, batch collection, Firecrawl, Playwright, Scrapling, Crawl4AI, Google Ads SERP, Facebook Ad Library, X/Twitter, Trustpilot, or job platform data collection.

Skill Index

The following index is node scripts/update-skill-index.js automatically generated, used to let Claude continue routing to the most relevant sub-skills after the top-level expert is triggered.

Instructions for Claude

1. First semantically match the user's current task with each sub-skill's 触发语义 , do not just look at the directory name.
2. Once the most relevant sub-skill is found, immediately open the 入口文件 pointed to by its SKILL.md, and use it as the entry point for the next layer.
3. After entering the sub-skill, then load the references/、scripts/、assets/in the same directory as needed according to the sub-skill's own instructions; do not pre-expand large sections of detail in the top-level expert.
4. If multiple sub-skills are relevant, first load the one closest to the main goal, then supplement with other sub-skills as needed, avoiding loading too much context at once.
5. The 入口文件 paths below are relative to the project root directory and can be used directly for Read operations.

Sub-skill Index

crawl4ai (1)

• crawl4ai-pipeline-builder
  • Trigger semantics: Use when you need to use Crawl4AI for multi-URL scraping, Markdown generation, structured extraction, CLI batch crawl, session reuse, content filtering, or LLM-free schema extraction pipelines. Suitable for scenarios such as "batch scrape multiple pages", "convert a documentation site to markdown", "extract structured data with a CSS schema", "build a reusable crawl pipeline", etc.
  • Entry file: .claude/skills/scraping-specialist/references/domains/crawl4ai/SKILL.md

facebook-ad-library (1)

• facebook-ad-library-scraper
  • Trigger semantics: Use when you need to scrape Facebook Ad Library ad creatives, download images and videos, extract transcript content, organize a brand ad creative library, or analyze competitor creative formats. Suitable for scenarios such as "scrape the Facebook ad library", "save a certain brand's ad creatives", "analyze ad video copy and creative styles", etc.
  • Entry file: .claude/skills/scraping-specialist/references/domains/facebook-ad-library/SKILL.md

firecrawl (1)

• firecrawl-web-extraction
  • Trigger semantics: Use when you need to use Firecrawl to complete web search, single-page scraping, structured extraction, screenshots, URL mapping, full-site crawl, or JavaScript-rendered page content extraction. Suitable for "search and scrape web content", "extract the main content of a certain URL", "extract web page fields according to a schema", "batch scrape documentation sites", and scenarios requiring the Firecrawl CLI.
  • Entry file: .claude/skills/scraping-specialist/references/domains/firecrawl/SKILL.md

general-web-scraping (1)

• web-scraping-playbook
  • Trigger semantics: Use when you need to formulate a scraping plan for any website, do site reconnaissance, discover sitemaps or APIs, choose the optimal scraping path, handle 403/Cloudflare/rate limiting, or upgrade scraping logic into a maintainable production solution. Suitable for scenarios such as "scrape this site", "first determine whether there is an API", "blocked by anti-scraping", "turn this scraping process into a continuously runnable scraper", etc.
  • Entry file: .claude/skills/scraping-specialist/references/domains/general-web-scraping/SKILL.md

ida-pro-skills (4)

• ida-domain-api
  • Trigger semantics: Analyze binaries using the Domain API for IDA Pro. Use when examining program structure, functions, disassembly, cross-references, or strings.
  • Entry file: .claude/skills/scraping-specialist/references/domains/ida-pro-skills/ida-domain-api/SKILL.md
• ida-domain-expert
  • Trigger semantics: Senior IDA Domain Python developer and IDA Pro reverse engineer. Use proactively when writing IDA Domain scripts, debugging IDA API issues, analyzing binary analysis problems, or when the user needs expert guidance on reverse engineering tasks with IDA Pro.
  • Entry file: .claude/skills/scraping-specialist/references/domains/ida-pro-skills/ida-domain-expert/SKILL.md
• ida-domain-scripting
  • Trigger semantics: Write and execute Python scripts using the IDA Domain API for reverse engineering. Analyze binaries, extract functions, strings, cross-references, decompile code, work with IDA Pro databases (.i64/.idb). Use when user wants to analyze binaries, reverse engineer executables, or automate IDA Pro tasks.
  • Entry file: .claude/skills/scraping-specialist/references/domains/ida-pro-skills/ida-domain-scripting/SKILL.md
• ida-plugin-development
  • Trigger semantics: Develop plugins for IDA Pro in Python, using idiomatic patterns, lessons, and tricks, including the Python Domain API (ida-domain). Use when creating both GUI (Qt) and background plugins for inspecting and rendering things program structure, functions, disassembly, cross-references, and strings.
  • Entry file: .claude/skills/scraping-specialist/references/domains/ida-pro-skills/ida-plugin-development/SKILL.md

job-platforms (1)

• taiwan-job-platform-scraper
  • Trigger semantics: Use when you need to scrape job listings from Taiwanese job platforms, aggregate job information from 104, CakeResume, and Yourator across sites, or organize job samples by keyword and location. Suitable for scenarios such as "search for frontend engineer jobs", "organize samples of the Taiwanese job market", and "compare results from multiple job platforms".
  • Entry file: .claude/skills/scraping-specialist/references/domains/job-platforms/SKILL.md

review-intelligence (1)

• trustpilot-review-intelligence
  • Trigger semantics: Use when you need to scrape Trustpilot reviews, analyze rating distributions, extract positive and negative themes, generate competitor reputation insights, or provide review intelligence for advertising and conversion copy. Suitable for scenarios such as "scrape Trustpilot reviews for a certain brand", "analyze negative review themes", "export reviews to CSV/JSON", and "do reputation competitor research"; the script is located at scripts/tpscraper.js。
  • Entry file: .claude/skills/scraping-specialist/references/domains/review-intelligence/SKILL.md

scrapling (1)

• scrapling-web-scraper
  • Trigger semantics: Use when you need to use Scrapling to automatically choose a scraping method based on site characteristics, bypass Cloudflare or WAF, handle logged-in pages, directly parse existing HTML, or quickly generate Python scraping scripts. Suitable for scenarios such as "this site has Cloudflare", "need to scrape after logging in", "give me a Scrapling script", and "batch scrape multiple pages while preserving sessions".
  • Entry file: .claude/skills/scraping-specialist/references/domains/scrapling/SKILL.md

search-ad-intelligence (1)

• google-serp-ad-intelligence
  • Trigger semantics: Use when you need to scrape Google search ads, analyze competitor ad copy, observe the Google Ads SERP by keyword and region, do PPC competitor intelligence, or collect ad pages at zero API cost. Suitable for scenarios such as "scrape the ads for these keywords in a certain region", "analyze how competitor ads are written", and "view Google Ads placements and extensions"; the execution script is located at scripts/scrape-ads-playwright.cjs。
  • Entry file: .claude/skills/scraping-specialist/references/domains/search-ad-intelligence/SKILL.md

social-intelligence (1)

• x-topic-intelligence
  • Trigger semantics: Use when you need to collect popular posts, trending discussions, KOL content, hook writing styles, reply-driven content, or social media intelligence datasets on X/Twitter around a certain topic. Suitable for scenarios such as content research, public opinion research, topic scanning, social media competitor observation, and content opportunity analysis; prioritize the fast mode, and perform multiple rounds of DOM scraping for in-depth collection.
  • Entry file: .claude/skills/scraping-specialist/references/domains/social-intelligence/SKILL.md

structured-extraction (1)

• structured-web-data-extractor
  • Trigger semantics: Use when you need to extract data from web directories, contacts, alumni directories, attendees, merchant lists, membership directories, company lists, or paginated list pages into CSV, JSON, or tables. Suitable for scenarios such as "scrape all records on this page into a table", "scrape a directory after logging in", "batch export directory page data", and "extract scroll-loaded lists"; prioritize using Playwright MCP.
  • Entry file: .claude/skills/scraping-specialist/references/domains/structured-extraction/SKILL.md

Notes

• Top-level SKILL.md It is only responsible for routing, and does not expand specific scraping scripts or lengthy tactical explanations here.
• First match sub-skills based on task semantics, then enter the corresponding directory to read on demand references/ or execute scripts/。
• When encountering an unfamiliar site, first enter the general scraping sub-skill, perform reconnaissance and strategy selection first, and then decide whether to switch to Firecrawl, browser automation, or vertical-scenario sub-skills.