google-image-crawler

Google Image Crawler

Playwright-based Google Image Crawler, supporting extraction of high-definition original image URLs and batch downloads.

Quick Start

Install Dependencies

pip install playwright aiohttp requests tqdm
playwright install chromium

Basic Usage

import asyncio
from crawler import GoogleImageCrawler

async def main(): async with GoogleImageCrawler() as crawler: # Search for images results = await crawler.search("cute cats", num_images=10)

    # 打印结果
    for result in results:
        print(f"URL: {result.url}")
        print(f"Size: {result.width}x{result.height}")
    
    # 下载图片
    downloaded = await crawler.download_images(
        results,
        output_dir="./downloads"
    )

asyncio.run(main())


### 同步 API（快速调用）

```python
from crawler import search_images_sync

# 一行代码搜索图片
results = search_images_sync("mountain landscape", num_images=5)
for r in results:
    print(r.url)

Core Features

1. Image Search (GoogleImageCrawler.search)

Use Playwright to automate Google Image search and extract original high-definition image URLs.

results = await crawler.search(
    keyword="search term",      # 搜索关键词
    num_images=10,              # 获取数量
    safe_search=True,           # 安全搜索
    min_width=None,             # 最小宽度过滤
    min_height=None             # 最小高度过滤
)

2. Image Download (GoogleImageCrawler.download_images)

Asynchronously batch download images, with support for concurrency control.

downloaded = await crawler.download_images(
    results=results,            # 搜索结果列表
    output_dir="./images",      # 输出目录
    concurrency=3               # 并发数
)

3. Standalone Download Module (ImageDownloader)

More powerful download functionality, with support for thread pool concurrency and progress display.

from downloader import ImageDownloader

downloader = ImageDownloader(
    output_dir="./downloads",
    concurrent=5,
    max_retries=3
)

results = downloader.download_batch(urls)

4. Command-Line Tool

# 从文件下载
python cli.py -f urls.txt -o ./images

# 带参数下载
python cli.py -f urls.txt -c 10 -t 60 -o ./images

# 下载单张图片
python cli.py -u "https://example.com/image.jpg" -o ./images

Usage Examples

Example 1: Basic Search

import asyncio
from crawler import GoogleImageCrawler

async def basic_search():
    async with GoogleImageCrawler(headless=True) as crawler:
        results = await crawler.search("sunset beach", num_images=5)
        
        print(f"找到 {len(results)} 张图片:")
        for i, result in enumerate(results, 1):
            print(f"{i}. {result.title[:40] if result.title else 'No title'}")
            print(f"   URL: {result.url[:70]}...")
            print(f"   尺寸: {result.width}x{result.height}")

asyncio.run(basic_search())

Example 2: HD Wallpaper Search (with size filtering)

async def hd_wallpaper_search():
    async with GoogleImageCrawler() as crawler:
        results = await crawler.search(
            "4k wallpaper",
            num_images=5,
            min_width=1920,
            min_height=1080
        )
        
        print(f"找到 {len(results)} 张高清图片:")
        for result in results:
            print(f"✓ {result.width}x{result.height} - {result.url[:60]}...")

asyncio.run(hd_wallpaper_search())

Example 3: Search and Download

async def search_and_download():
    async with GoogleImageCrawler() as crawler:
        # 搜索
        results = await crawler.search("puppy", num_images=10)
        print(f"搜索完成，准备下载 {len(results)} 张图片...")
        
        # 下载
        downloaded = await crawler.download_images(
            results,
            output_dir="./downloads",
            concurrency=2
        )
        print(f"下载成功: {len(downloaded)} 张")
        for path in downloaded:
            print(f"  → {path}")

asyncio.run(search_and_download())

Example 4: Use the Standalone Download Module

from downloader import ImageDownloader

# 单独使用下载模块
downloader = ImageDownloader(
    output_dir="./downloads",
    concurrent=5,
    timeout=30,
    max_retries=3
)

# 从 URL 列表下载
urls = [
    "https://example.com/image1.jpg",
    "https://example.com/image2.jpg"
]

results = downloader.download_batch(urls)

# 统计结果
success = len(results['success'])
failed = len(results['failed'])
print(f"成功: {success}, 失败: {failed}")

Example 5: Combined Usage (Recommended)

import asyncio
from crawler import GoogleImageCrawler
from downloader import ImageDownloader

async def combined_workflow():
    # 1. 爬取图片 URL
    async with GoogleImageCrawler() as crawler:
        results = await crawler.search("mountain", num_images=20)
        urls = [r.url for r in results]
    
    # 2. 使用 Downloader 下载（支持更多功能）
    downloader = ImageDownloader(
        output_dir="./mountain_images",
        concurrent=5,
        max_retries=3
    )
    
    with downloader:
        results = downloader.download_batch(urls)
        
        # 统计结果
        success = len(results['success'])
        failed = len(results['failed'])
        print(f"成功: {success}/{success + failed}")

asyncio.run(combined_workflow())

Parameter Description