diagnosing-failed-warehouse-syncs
Diagnose why a data warehouse sync is failing and recommend the right recovery action. Use when the user asks "why isn't my Stripe/Postgres/Hubspot sync working?", "this table has been stuck for hours", "the data in the warehouse looks wrong", or wants to troubleshoot a specific source or schema. Covers source-level vs schema-level failures, stuck Running states, credential and schema-drift errors, incremental-field misconfig, CDC prerequisite failures, and the cancel / reload / resync / delete-data recovery actions.
How do I install this agent skill?
npx skills add https://github.com/posthog/ai-plugin --skill diagnosing-failed-warehouse-syncsIs this agent skill safe to install?
- Gen Agent Trust Hubpass
The skill is designed to diagnose data warehouse sync failures and incorporates several security best practices, such as using secure browser-based credential handling. However, it ingests error messages from external sources which represents a potential surface for indirect prompt injection given the skill's destructive capabilities.
- Socketpass
No alerts
- Snykpass
Risk: LOW · No issues
What does this agent skill do?
Diagnosing failed data warehouse syncs
Work top-down when a data warehouse source or table is failing, stuck, or producing bad data: source → schema → recovery action. Do not jump straight to "resync from scratch" — that discards synced data and restarts from zero, which is rarely the right first step.
When to use this skill
- The user reports a specific sync is failing (e.g. "my Stripe source is red")
- A table has been in
Runningstate far longer than expected - Data in a warehouse table is stale, missing rows, or looks corrupt
- Latest rows aren't appearing despite the schema being marked
Completed - The user is choosing between cancel / reload / resync / delete-data and isn't sure which
- Another skill — typically
auditing-warehouse-source-health— has surfaced a failing source or schema and the user wants to dig into it
Both entry points (user-reported and audit-handoff) use the same workflow; the audit just means you already know which item to diagnose and can skip Step 1's discovery search.
Available tools
| Tool | Purpose |
|---|---|
external-data-sources-list | List all sources with connection status and latest error |
external-data-sources-retrieve | Full details for one source including all its schemas |
external-data-schemas-list | All table schemas across all sources, with per-table status + latest_error |
external-data-schemas-retrieve | Full details for one schema including sync_type_config |
external-data-schemas-cancel | Cancel a sync currently in Running state |
external-data-schemas-reload | Trigger a sync using the configured sync method (respects incremental) |
external-data-schemas-resync | Full resync — wipes synced data and restarts. Destructive |
external-data-schemas-delete-data | Delete the synced table but keep the schema entry |
external-data-schemas-partial-update | Change sync_type / incremental_field / cdc_table_mode |
data-warehouse-source-connect-link | Secure browser link where the user re-enters a source's credentials |
data-warehouse-stored-credentials-list | Find the credential_id the user stored on the connect page |
external-data-sources-partial-update | Reconnect a source with {"payload": {"credential_id": ...}} |
external-data-sources-reload | Retrigger syncs for every enabled schema on a source |
external-data-sources-refresh-schemas | Re-fetch the source's table list to pick up new tables |
external-data-sources-check-cdc-prerequisites-create | Verify Postgres CDC setup for a source |
external-data-sources-cdc-status-retrieve | Live CDC health: slot, publication, lag, paused schedule |
external-data-sources-resume-cdc-create | Unpause CDC after a fixed failure, no re-snapshot |
external-data-sources-repair-cdc-create | Recreate a lost slot or publication. Forces a full re-sync |
external-data-schemas-incremental-fields-create | Refresh candidate incremental fields when the source schema has changed |
external-data-sources-webhook-info-retrieve | Check webhook registration state and external service status |
external-data-sources-create-webhook-create | Re-register a webhook that was lost or never registered |
external-data-sources-update-webhook-inputs-create | Update the signing secret after rotation on the source side |
external-data-sources-delete-webhook-create | Remove a broken webhook before re-registering |
platform-alerts-list | Check whether a sync failure alert already exists |
platform-alerts-create | Create a sync failure alert for a source or its tables |
warehouse-alerts-destinations-create | Choose where the alert notifies (Slack, webhook, and so on) |
Workflow
Step 1 — Locate the failing item
If the user named a source, go straight to external-data-sources-retrieve. Otherwise start with
external-data-sources-list and external-data-schemas-list to find what's red.
Two kinds of failure:
- Source-level (
ExternalDataSource.status = "Error"): the connection itself is broken — credentials expired, host unreachable, account disabled. Affects every table. - Schema-level — the source connects fine but one or more tables are failing. In the serialized API response
from
external-data-schemas-list, look forstatusvalues"Failed","Billing limits", or"Billing limits too low". (The underlying model enum values areBillingLimitReachedandBillingLimitTooLow, but the serializer rewrites them — match on both the human-readable and enum forms to be safe.)
A source can look Completed at the top level while one of its schemas is Failed — always check both.
Step 2 — Classify the schema status
From external-data-schemas-list, each schema has a status:
| Status | Meaning | Usually means |
|---|---|---|
Running | Sync currently executing | Normal, unless stuck for hours |
Completed | Last sync finished successfully | Healthy |
Failed | Last sync errored — see latest_error | Needs diagnosis |
Paused | User disabled sync (should_sync = false) | Intentional |
Billing limits (serializer) / BillingLimitReached (enum) | Team hit its warehouse row quota | Billing issue, not a technical failure |
Billing limits too low (serializer) / BillingLimitTooLow (enum) | Team has insufficient credit | Billing issue |
Always check last_synced_at alongside status. A schema in Running with last_synced_at from 12 hours ago is
almost certainly stuck, even though the status isn't Failed.
Step 3 — Interpret latest_error
Map the latest_error string to a root cause. Common patterns:
| Error substring | Root cause | Fix |
|---|---|---|
authentication failed, 401, 403, invalid credentials | Credentials expired or rotated | Reconnect the source with a connect link (see B) |
Could not establish session to SSH gateway, Could not connect to your SSH tunnel | SSH tunnel misconfigured or remote host down | User checks SSH host/key/bastion |
Primary key required for incremental syncs | Table has no PK and sync_type is incremental/cdc | Either add PK in source, or switch schema to full_refresh |
primary keys for this table are not unique | Declared PK columns aren't actually unique | Pick different PK columns via partial-update |
Integration matching query does not exist | Source's saved integration was deleted | Reconnect the source with a connect link (see B) |
column "X" does not exist, does not have a column named | Schema drift — incremental field or tracked column removed | Use incremental-fields-create to re-detect, then partial-update |
relation "..." does not exist | Source table was dropped/renamed | Remove schema or rename source-side |
SSL, connection refused, timeout, unreachable | Network / firewall / host reachability | User side — check host/port/allowlist |
replication slot, publication, wal_level | CDC prerequisites broken | Run check-cdc-prerequisites-create; may need slot recreate |
Schema exceeds row limit, billing | Billing limit | Upgrade plan or disable the schema |
If latest_error is null but the schema is Failed, retrieve the schema directly — the error may only be populated
on the detail view.
Step 4 — Pick the recovery action
The recovery action depends on root cause, not just status. Match the user's situation to one of these:
A. Transient failure (network blip, temporary API outage)
- Data synced so far is still valid.
- Action:
external-data-schemas-reloadto retry using the configured sync method. - Incremental/append syncs pick up where they left off.
B. Credentials expired or rotated
- Every schema under the source is failing with an auth error, or the OAuth token expired.
- Never ask the user to paste a password, API key, or token into the chat. Reconnect the source this way:
- Call
data-warehouse-source-connect-linkwith the source'ssource_typeand share theconnect_url. The user enters new credentials or authorizes OAuth again in their browser. The page does not change the source. - When the user confirms they are done, call
data-warehouse-stored-credentials-listwith the samesource_typeand take the newestcredential_id. - Call
external-data-sources-partial-updatewith the source id and{"payload": {"credential_id": "<id>"}}. Do not sendjob_inputsin the same call. PostHog replaces the source's connection details with the stored ones and checks the connection before it saves. An SSH tunnel the user did not enter on the page is turned off. A failed update keeps the stored credential. When the connection check fails, get a new link so the user can enter correct details.
- Call
- The stored credential is single-use and expires after 24 hours. Only the user who filled the page can use it, so that user must be the one this session authenticates as.
- Then the reload happens automatically when the source status flips back to running, or trigger it manually with
external-data-sources-reload.
C. Schema drift — column renamed, dropped, or type changed
- Error mentions a specific column that no longer matches the source.
- Action:
external-data-schemas-incremental-fields-createto get the current fields, thenexternal-data-schemas-partial-updatewith the correctedincremental_field/incremental_field_type/primary_key_columns. Usually no need to wipe data.
C2. Added / renamed tables in the source database
- User mentions "I added a new table to Postgres but it isn't appearing", or a source table was renamed.
- Action:
external-data-sources-refresh-schemasto pick up the new table list, then configure sync on any new schemas.
D. Incremental state is wrong (duplicates, missing rows, data looks corrupt)
- Schema status may be
Completed— this isn't a "failure" per se, it's bad data. - Action:
external-data-schemas-resyncto wipe synced data and re-import from source. Destructive but often the right call for data-quality issues.
E. CDC pipeline broken on Postgres
- Error mentions replication slot, publication, WAL.
- Action: call
external-data-sources-cdc-status-retrievefirst. It reads the source database, so a 400 means the source is unreachable: fix the credentials first. schedule_paused=truewithslot_existsandpublication_existsboth true: a failure paused extraction and nothing was lost. Fix the cause, then callexternal-data-sources-resume-cdc-create. It does not re-snapshot.slot_exists=falseorpublication_exists=false: callexternal-data-sources-repair-cdc-create. It recreates them and resets every CDC table to a full re-sync, so ask the user first. It refuses when CDC looks healthy.- Use
external-data-sources-check-cdc-prerequisites-createto list setup problems on the Postgres side (wal_level, permissions). Then callexternal-data-schemas-reload.
F. Sync is stuck in Running for hours
- Check
last_synced_at. If it's hours old and stillRunning, the job is orphaned. - Action:
external-data-schemas-cancelto stop it, thenexternal-data-schemas-reload.
G. Table data is corrupt but you want to keep the schema config
- Action:
external-data-schemas-delete-datato drop the synced table but preserve the schema entry. It also turns syncing off, so setshould_sync: truewithexternal-data-schemas-partial-update, then reload. The reload re-imports from scratch without losing the configured sync_type/incremental_field.
H. Billing limit
- Action isn't technical. Explain the limit, recommend upgrading the plan or disabling lower-priority schemas so the important ones fit under quota.
I. Webhook-backed schema isn't receiving events
- Symptoms: schema has
sync_type: "webhook", initial bulk sync finished, but no new rows arrive despite activity on the source side. Status may still readCompletedbecause the bulk sync (the safety-net cadence) is succeeding — the problem is the push path. - Action:
external-data-sources-webhook-info-retrieve({id}).- If
exists: false→ the webhook was never registered, or was deleted. Callcreate-webhook-createto register it. - If
exists: truebutexternal_status.erroris set → typically "API key doesn't have permission to read webhooks" or similar. The webhook may have been deleted on the source's dashboard. Re-create it. - If
external_status.statusisn't"enabled"→ the source disabled the webhook (usually after repeated delivery failures). Re-enable or re-register. - If payloads are arriving but failing signature verification → the signing secret was rotated. Get the new
one from the source's dashboard and call
update-webhook-inputs-create({id}, {inputs: {signing_secret: "..."}}).
- After any fix, check the source's webhook logs (on their side) to confirm PostHog is now responding 2xx.
Step 5 — Confirm before destructive actions
Three recovery actions discard data and cannot be undone:
external-data-schemas-resync— wipes synced rows, re-imports from scratchexternal-data-schemas-delete-data— drops the synced tableexternal-data-sources-destroy— deletes the source and all its schemas
Always present the fix you're proposing and wait for explicit approval before calling any of these. "Just try resync" is rarely the right default.
Step 6 — Offer a sync failure alert after the fix
After the sync works again, offer to alert the user the next time it breaks, so they do not need to open PostHog to find out. Offer it once, and do not create the alert unless the user agrees.
- Call
platform-alerts-listwithsource_kind=warehouse_source. If an enabled alert already watches this source (itssource_config.source_idsnames it, or is empty) and has a destination (destination_typesis not empty), say so and stop. If an alert exists with no destination, offer to add one. - Ask where messages go: a Slack channel, a webhook, Microsoft Teams, Discord, or PagerDuty. Do not guess the channel.
- Call
platform-alerts-createwithsource_kind=warehouse_source, a descriptivename, andsource_config.source_idsset to the source ID. Usegroup_by=schemaandgrouping.keys=["schema_id"]for one message per failing table, or the defaultgroup_by=sourceandgrouping.keys=["source_id"]for one message per source. - Call
warehouse-alerts-destinations-createwith theidthe create call returned. An alert with no destination notifies nobody.
If the tools are not available, the user's project may not have sync failure alerts yet. Say so and skip this step.
Example interaction
User: "Our Stripe sync is broken, can you check?"
Agent:
- external-data-sources-list → find Stripe source, status = Error
- external-data-sources-retrieve({id}) → latest_error: "authentication failed: 401 Unauthorized"
- Report: "Your Stripe source's API key is no longer authenticating.
All 8 tables under it are failing with 401s. This usually means the key was rotated on the Stripe side.
To fix:
1. Grab a fresh restricted API key from the Stripe dashboard.
2. Enter it on this secure page: <connect_url>. Don't paste it here.
3. I'll reconnect the source, and syncs will resume — no data loss.
Tell me when you're done."
(The agent got <connect_url> from data-warehouse-source-connect-link({source_type: "Stripe"}).)
User: "Done"
Agent:
- data-warehouse-stored-credentials-list({source_type: "Stripe"}) → newest credential_id
- external-data-sources-partial-update({id}, {payload: {credential_id: "<credential_id>"}})
- external-data-sources-reload({id}) to trigger retry
- Report: "Updated and re-triggered. Check back in a few minutes — latest_error should clear."
Important notes
- Source status overrides schema status for diagnosis. If the source is
Error, nothing under it will work; fixing the source usually fixes all its schemas at once. Runningisn't always healthy. Cross-checklast_synced_at. A sync stuck inRunningneedscancelthenreload, notresync.- Resync is destructive. It discards synced data. Only recommend it when the data itself is bad (duplicates, missing rows, corrupt) or when recovery genuinely requires a clean slate (lost WAL position on CDC). Never use it as a first-try for transient errors.
- Delete-data preserves config. When a user says "I just want to start this table over from scratch", prefer
delete-data+reloadoverresync+ new schema entry — it keeps the configured sync_type / incremental_field / PK setup.delete-dataturns syncing off, so setshould_sync: truebefore the reload. - A failure can belong to a destination, not the source. Some projects sync a source to their own database as
well as to PostHog. When they do,
latest_erroris prefixed with the destination's name, e.g.customer postgres: connection refused. That is the customer's database refusing the write, not the source failing to extract — check their database and its credentials rather than the source's. Every destination shares one lifecycle, so one unreachable destination holds up the whole sync, PostHog included. - Billing limits aren't technical failures. Don't try to retry or reconfigure your way out. Route to billing.
- Webhook failures can hide behind a green status. A webhook-type schema whose bulk fallback sync succeeded looks
Completedeven when the push channel is broken. When users say "my data is hours behind" on a webhook schema, callwebhook-info-retrievebefore looking at schema status. Webhook issues don't surface onexternal-data-schemas-list.
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/posthog/ai-plugin/diagnosing-failed-warehouse-syncs">View diagnosing-failed-warehouse-syncs on skillZs</a>