doris-debug-import

Import / Group Commit

async_mode WAL semantics

Write WAL → RPC returns → background group commit → delete_wal on success
(group_commit_mgr.cpp). Drain rate = commit finish rate ≠ compaction GB/s.

Knobs (config.cpp)

Config	Default	Role
group_commit_insert_threads	10	commit worker pool
group_commit_relay_wal_threads	10	WAL replay
group_commit_wal_path	under storage_root	WAL disks
group_commit_wal_max_disk_limit	10%	WAL quota
table group_commit_data_bytes	often 64MB	size flush
table group_commit_interval_ms	often 10000	time flush

High MB/s guidance

1. Split load across BEs (WAL is local to coordinator BE).
2. wal-du on each BE storage path.
3. Prefer dedicated NVMe for group_commit_wal_path.
4. A/B group_commit_data_bytes (128MB vs 512MB) — size threshold dominates when ingest ≫ interval.
5. sync_mode or client throttle for backpressure.

./scripts/doris-debug wal-du /path/to/be/storage
./scripts/doris-debug log-grep be/log --pack group_commit