doris-debug-query

Query Troubleshooting

Causes

ID	Cause	Evidence
A	Client timeout too small	`query_timeout` vs Profile wall
B	Planner	`Nereids cost too much time` / `nereids_timeout_second`
C	BE compute (scan/join/spill)	Profile active ExecTime
D	Exchange / brpc	WaitForData ≈ timeout; `failed to send brpc when exchange`; `[E1008]` on `:8060`

10 min

SET enable_profile = true;
-- reproduce

./scripts/doris-debug log-grep fe/log be/log --query-id "$QID" --pack planner
./scripts/doris-debug log-grep be/log --query-id "$QID" --pack exchange
./scripts/doris-debug audit-topk fe/log/fe.audit.log -k 20 --min-ms 5000

Planner mitigation (session only): SET disable_join_reorder=true; for fat ETL joins; fix stats with ANALYZE.

Cause C — BE compute (scan/join/spill)

When Profile shows high active ExecTime (not Wait*), route to doris-profile-reader for deep Profile analysis:

Skill routing: If doris-profile-reader is available, invoke it with the query's Profile to identify which operator (scan/join/agg/spill) dominates. Key indicators to pass:

ActiveTime per operator

RowsRead / ScanBytes for scan operators

SpillDataSize if disk spill occurred

JoinType + JoinRows for join operators

Without profile-reader, check directly:

# Get the Profile text
curl -s "http://$FE:8030/api/query_profile?query_id=$QID"

Key Profile counters for Cause C:

Counter	Meaning	Threshold
`ScannerGetBlockTime`	Time reading from storage	> 50% of ExecTime → IO bound
`SpillDataSize`	Data spilled to disk	> 0 → memory pressure, check `enable_spill`
`JoinProbeTime`	Hash join probe wall time	> 30% → check join order / type
`RowsRead` / `ScanBytes`	Scan volume	> 1B rows → check partition pruning

Profile interpretation: prefer active time over Wait*.

Cause D — Exchange / brpc

Code facts (apache/doris):

Error text in be/src/exec/operator/exchange_sink_buffer.h.
GET /api/reset_rpc_channel/{endpoints} clears brpc_internal_client_cache only (reset_rpc_channel_action.cpp).
enable_brpc_connection_check (default false) → FragmentMgr::_check_brpc_available hand_shake; consecutive failures erase internal cache.

curl -s "http://$BE:8040/api/reset_rpc_channel/all"

# be.conf — restart for static DEFINE_Bool
enable_brpc_connection_check = true
brpc_connection_check_timeout_ms = 10000

Do not trust ICMP alone. Check priority_networks.

Source

SessionVariable.java — QUERY_TIMEOUT, NEREIDS_TIMEOUT_SECOND
Profile WaitForData — exchange source operator
exchange_sink_buffer.h, reset_rpc_channel_action.cpp, fragment_mgr.cpp