Fix "Merges are processing significantly slower than inserts"

Code: 252. DB::Exception: Too many parts (N). Merges are processing significantly slower than inserts is the same error as Too many parts, but this page is for the other side of it: your write rate has outpaced the background merge scheduler, not that the insert pattern itself is wrong. If batching and partitioning already look sane, the bottleneck is merge capacity.

Why it happens

• Undersized merge pool. background_pool_size / background_merge_scheduling_pool_size too small for the sustained write rate — not enough worker threads to merge parts as fast as they arrive.
• Slow disks. Merges are I/O-heavy (read every source part, write the merged part). Network-attached storage without enough IOPS, or spinning disks under a high write rate, can't sustain it.
• Very large parts. max_bytes_to_merge_at_max_space_in_pool (default 150 GiB) lets merges grow large; a single big merge can occupy a worker for a long time and delay everything queued behind it.
• Concurrent mutations. ALTER … UPDATE/DELETE runs through the same merge pool and competes with regular merges for threads.
• A single hot partition serializing merges that could otherwise run in parallel across partitions.

Diagnose

See what's merging right now, then look at the trend:

-- Active merges: what's running, how far along, how long it's taken
SELECT
    database,
    table,
    round(progress * 100, 2) AS pct_done,
    elapsed,
    is_mutation,
    formatReadableSize(total_size_bytes_compressed) AS merge_size
FROM system.merges
ORDER BY elapsed DESC;

-- Merge count and duration over the last hour — a rising queue means merges are losing ground
SELECT
    toStartOfMinute(event_time) AS minute,
    countIf(event_type = 'NewPart') AS parts_created,
    countIf(event_type = 'MergeParts') AS parts_merged
FROM system.part_log
WHERE event_time > now() - INTERVAL 1 HOUR
GROUP BY minute
ORDER BY minute;

parts_created consistently outrunning parts_merged confirms the backlog is on the merge side, not a one-off spike.

Fix

See it in chmonitor

The Operations feature's Merges and Merge performance pages show active merges and historical duration trends live, so a growing backlog is visible before it throttles inserts. Try it on the live demo.

Related