playwright_browser

Playwright Browser Automation Skill

This skill provides complete browser automation capabilities based on Playwright, supporting complex web interaction tasks in headless mode.

Core Features

1. Basic Browsing (browser_tool.py)

Visit web pages, take screenshots, and read HTML content.

• Supports waiting for specific elements to finish loading
• Supports full-page screenshots and HTML content preview
• Automatically handles sandbox restrictions in container environments

2. Data Scraping (scraper.py)

Use CSS selectors to precisely extract web page data.

• Supports single and multiple element extraction
• Supports extracting text or attribute values
• JSON format output for easy subsequent processing

3. Search Function (search.py)

Use the Baidu search engine and return structured results.

• Automatically fill in keywords and submit the search
• Extract titles, links, and descriptions
• Supports custom result counts

4. Automated Interaction (automator.py)

Execute complex action chains and simulate real user behavior.

• Click, input, dropdown selection
• Scroll, wait, execute JavaScript
• Supports screenshots at key steps

5. Page Snapshot (snapshot.py)

High-quality page screenshots and PDF export.

• Full-page or viewport screenshots
• PDF format export
• Automatically generate thumbnails

6. Batch Image Download (download_images.py)

Automatically extract and download all images from a web page.

• Intelligently extract all img tags
• Generate filenames based on alt or title
• Supports custom download counts and output directories
• High-quality original image downloads

Usage

Basic Browsing

Visit a web page and take a screenshot: ```bash python skills/playwright_browser/browser_tool.py --url https://example.com --screenshot --output example.png ```

Read page HTML content: ```bash python skills/playwright_browser/browser_tool.py --url https://example.com --read ```

Wait for a specific element to load: ```bash python skills/playwright_browser/browser_tool.py --url https://example.com --screenshot --wait .main-content ```

Data Scraping

Extract text from a single element: ```bash python skills/playwright_browser/scraper.py https://example.com --selector title:h1:text ```

Extract multiple elements (such as all links): ```bash python skills/playwright_browser/scraper.py https://example.com --selector links:a:href:multiple --json ```

Extract specific attributes: ```bash python skills/playwright_browser/scraper.py https://example.com --selector img_url:img:src ```

Baidu Search

Search keywords: ```bash python skills/playwright_browser/search.py "搜索关键词" --max 10 ```

JSON format output: ```bash python skills/playwright_browser/search.py "搜索关键词" --json > results.json ```

Automated Interaction

Use a JSON configuration file: ```bash python skills/playwright_browser/automator.py https://example.com --file actions.json ```

Pass in the action chain directly: ```bash python skills/playwright_browser/automator.py https://example.com --actions '[{"type":"fill","selector":"#input","value":"test"},{"type":"click","selector":"#submit"}]' ```

actions.json example: ```json [ {"type": "wait", "delay": 2000}, {"type": "fill", "selector": "#username", "value": "alice"}, {"type": "fill", "selector": "#password", "value": "password"}, {"type": "click", "selector": "#login"}, {"type": "wait_for", "selector": ".dashboard"}, {"type": "screenshot", "filename": "dashboard.png"} ] ```

Page Snapshot

Full-page screenshot: ```bash python skills/playwright_browser/snapshot.py https://example.com --output fullpage --full-page ```

Export as PDF: ```bash python skills/playwright_browser/snapshot.py https://example.com --output report --format pdf ```

Batch Image Download

Download all images from a web page: ```bash python skills/playwright_browser/download_images.py https://example.com --output-dir downloaded_images --max 10 ```

Parameter description:

• --output-dir: Output directory (relative to alice_output/), defaults to "images"
• --max: Maximum download count, defaults to 20

Example: ```bash

Download up to 5 images from the NVIDIA official website

python skills/playwright_browser/download_images.py https://www.nvidia.cn --output-dir nvidia_images --max 5 ```

Technical Details

• Browser Engine: Chromium (headless mode)
• Viewport Setting: Default 1280x800 (adjustable as needed)
• Container Optimization: Automatically add --no-sandbox and --disable-setuid-sandbox parameters
• Asynchronous Execution: Use asyncio to achieve efficient concurrency
• Output Directory: All files are saved by default to alice_output/

Supported Operation Types (automator.py)

Type	Parameters	Description
click	selector	Click the specified element
fill	selector, value	Input text
select	selector, value	Dropdown selection
wait	delay	Wait for the specified number of milliseconds
scroll	selector or position	Scroll to the element or the top/bottom of the page
screenshot	filename	Save screenshot
evaluate	script	Execute JavaScript
wait_for	selector	Wait for the element to appear

Selector Configuration Format (scraper.py)

Selector parameter format:name:css_selector:attribute[:multiple]

• name: The key name of the extraction result
• css_selector: CSS selector
• attribute: The attribute to extract, by default text(text content)
• multiple: Optional, set to true to extract all matching items

Example:

• title:h1:text - Extract the text of the h1 tag
• images:img:src:multiple - Extract the src attribute of all img elements
• links:a:href - Extract the href of the first a tag

Notes

• Container Restrictions: When running in a container environment, you must use the --no-sandbox parameter (already handled automatically)
• Timeout Settings: The default timeout is 30 seconds, which can be adjusted according to task complexity
• Dynamic Content: If you need to wait for JavaScript rendering, use the --wait parameter or the wait_for operation
• Anti-Scraping Strategies: Some websites may have anti-scraping mechanisms; it is recommended to add appropriate delays
• Memory Usage: Pay attention to memory usage when processing a large number of pages; process in batches if needed
• Dependency Requirements: Requires playwright package and browser binaries to be installed
• Image Download: Some websites may have hotlink protection mechanisms, causing some images to fail to download

Dependency Installation

```bash pip install playwright playwright install chromium ```

Frequently Asked Questions

Q: How do I handle pages that require login? A: Use the operation chain of automator.py to execute fill (username) → fill (password) → click (login button) → wait_for (element after login) in sequence

Q: How do I extract content dynamically generated by JavaScript? A: Use the --wait parameter of browser_tool.py or the --wait parameter of scraper.py to wait for the element to load

Q: How do I handle pop-up dialogs? A: Use the evaluate operation of automator.py to execute JavaScript to close them, or disable them in advance in the page context

Q: Why are some websites inaccessible? A: It may be due to anti-scraping restrictions. You can try setting a user-agent or using a proxy (you need to modify the code to add context configuration)

Q: Why were only some images downloaded? A: Some images may have loading delays, hotlink protection, or base64 encoding. The script will automatically skip inaccessible images

Q: How are downloaded image filenames determined? A: The script will prioritize the alt attribute of the img tag, followed by the title attribute. If neither is present, it will use the format "image_N".