PeerDB Monitoring

CHM includes an optional, view-only PeerDB section (Mirrors and Peers) that surfaces replication status, throughput, lag, and per-mirror detail from a PeerDB deployment. CHM never mutates PeerDB — it proxies a read-only allowlist of the PeerDB REST API.

Further reading: see the blog post for why CDC pipelines need this kind of monitoring — lag, replication-slot growth, and batch failures — and a walkthrough of the views below.

What you can see

Configure

PEERDB_API_URL=https://peerdb.example.com/api
PEERDB_PASSWORD=your-peerdb-ui-password

PEERDB_API_URL=http://localhost:8113

Connection status

The header shows a status pill that distinguishes:

• Connected — API reachable and authenticated.
• Auth failed — credentials rejected (check PEERDB_PASSWORD). For the PeerDB UI this is the UI login password.
• Unreachable — wrong PEERDB_API_URL or a network/DNS issue.
• Not configured — PEERDB_API_URL is unset.

Security

Local development (mock)

To preview the full UI without a real PeerDB instance, run the bundled mock server:

pnpm run peerdb:mock                                  # serves :8113
PEERDB_API_URL=http://localhost:8113 pnpm run dev     # → /peerdb

Large mirror fleets

PeerDB's only index on flow_errors is flow_name, so the mirror-logs endpoint filtered by level=ERROR scans every error row for that mirror. On a fleet with a multi-million-row flow_errors (QRep writes most of it at info) that read is expensive and frequently returns nothing useful.

To make it cheap, add a composite index to the PeerDB catalog Postgres:

CREATE INDEX CONCURRENTLY flow_errors_flow_name_error_type_idx
  ON flow_errors (flow_name, error_type);

Without it, per-mirror ERROR-log reads degrade from a few hundred milliseconds to seconds, and enough of them in parallel will time out.

The alert sweep bounds its own load regardless of the index:

• it reads ERROR logs only for mirrors whose status is not STATUS_RUNNING, or that report an errorMessage — a healthy running mirror costs no log read;
• it runs at most PEERDB_SWEEP_CONCURRENCY reads at a time;
• it stops starting new work after PEERDB_SWEEP_BUDGET_MS, deferring the remaining mirrors to the next tick rather than overrunning the sweep.

On a 72-mirror fleet this turns ~100 log reads per tick into zero when the fleet is healthy.

Partial coverage

The alert sweep reads every mirror PeerDB lists. There is no hidden ceiling on fleet size: concurrency and wall-clock budget limit how fast it reads, not which mirrors it is allowed to read, and the mirrors most likely to alert are the ones it reads first.

A tick that could not read the whole fleet — because PEERDB_SWEEP_BUDGET_MS elapsed, or because PEERDB_SWEEP_MAX_MIRRORS is set — says so rather than reporting a clean result over a fleet it never looked at:

• the sweep summary reports the mirrors checked out of the mirrors listed, and flags the run as partial;
• an alert_events audit row with decision peerdb-coverage:partial records the counts, e.g. 50 of 72 mirrors checked — partial result: 22 unchecked.

A partial run is a coverage fact, not an incident: nothing is dispatched for it and no mirror's alert state is changed. The mirrors that were read are still classified normally. Mirrors left unread are retried on the next tick, so a fleet too large for one tick needs either a larger PEERDB_SWEEP_BUDGET_MS or a smaller fleet — not a cap.

Troubleshooting

Related

Blog: Monitoring PeerDB: snapshot progress, batch history, fleet lag, and slot health