Syncle

Troubleshooting

The failures that come up most often, and what each one means. Most turn out to be a database setting rather than a bug in Syncle. CDC in particular needs the source server configured for it, and no tool can turn those settings on from the outside.

syncle up exits without starting

Two known causes. If the message mentions resolving a reference, you are on an install older than 1.2.0, where the installer asked for the image by its release tag (v1.2.0) while images publish without the prefix (1.2.0). Reinstalling picks up the fixed script.

If the machine is offline, upgrade to 1.2.0 or later. Before it, a failed image pull aborted the whole start, so a host with the images already cached could not run Syncle at all; the pull is now best-effort.

Otherwise it is usually the port. Syncle publishes one, 3002, and refuses to start if something already holds it:

lsof -i :3002

The setup form wants a token

syncle up normally reads the first-run token off the server and opens the interface with it already accepted. Opening Syncle from a different device skips that, because the token is only readable by something with access to the container. Print it and paste it in:

syncle logs api

If the account already exists, there is no token to find — the API deletes it the moment setup succeeds, and clears it at boot when an account is present. Use the login form instead.

I cannot sign in: the password is gone

There is no e-mail address on file to send a link to, so the reset works the way first-run setup did: a code is printed on the server, and reading it proves you have access to the machine.

syncle reset-password

…or choose Forgot your password? on the sign-in screen, press the button, and read the code from syncle logs api (without the launcher: the API's console, or the file reset-code in its data directory). Enter it with a new password. The code works once, for fifteen minutes, and dies after ten wrong guesses; setting the password signs you in and ends every session there was. Pressing the button tells a stranger nothing — not even whether an account exists — and without access to the server the code it produces is a line in a log they cannot read. At most one code a minute is made, so the button cannot flood the log either.

Signing in (or saving anything) answers "came from another site"

A request that changes something is only taken from the app itself. A 403 with that message means the API could not tell that it was: the browser is too old to say so (Sec-Fetch-Site), and the address in the browser's bar is not the one the API was reached under — a reverse proxy that replaces the Host header is the usual cause. Either have the proxy pass it on (nginx: proxy_set_header Host $host;) or set WEB_ORIGIN to the public address, for example https://syncle.example.com. The origin it saw is in the message.

A CDC bridge never delivers anything

Almost always the source server is not configured for change data capture. The builder checks this before it lets you save, and names the setting that is missing — if you skipped past that, re-open the bridge and look at the trigger step. What each engine needs:

PostgreSQL

wal_level=logical, set in postgresql.conf or your provider's parameter group, followed by a server restart. This is the one prerequisite that cannot be automated, because it needs the restart.

If the bridge will not start and the message mentions a replica identity, the table has no primary key. PostgreSQL can then only report inserts for it — and publishing updates or deletes for such a table would make UPDATE and DELETE on it fail in your own database, which is why Syncle refuses instead of going ahead. Capture inserts only, add a primary key, or run ALTER TABLE your_table REPLICA IDENTITY FULL;. A refusal that says deletes cannot reach a target is the same rule seen from the other side: a delete carries only the source's key, so the target has to be keyed on it. Both are explained under what the table needs.

MySQL and MariaDB

Row-based binary logging, in my.cnf: log_bin=ON, binlog_format=ROW, binlog_row_image=FULL, and a unique server_id. It needs a server restart. On managed MySQL — RDS, Aurora, Cloud SQL — set these in the parameter group and reboot.

MongoDB

Change streams require a replica set. A single-node replica set is fine for development: start mongod with --replSet rs0 and run rs.initiate() once. Atlas already satisfies this.

Redis

Keyspace notifications, which Syncle tries to enable itself on start:

CONFIG SET notify-keyspace-events EA

Managed Redis usually refuses that and wants it enabled in the provider console instead. It can also be set permanently in redis.conf.

A watch bridge delivers nothing

A new watch bridge starts from now by default, so it ignores everything already in the table and only delivers rows that appear after it started. If you wanted the existing rows too, either set it to start from the beginning, or run a replay job once to backfill and leave the watch running for what follows.

If it delivers nothing even for new rows, check the cursor matches the table. A timestamp cursor on a column the application does not update will never advance; a table whose primary keys are UUIDs needs the primary-key diff strategy rather than an incrementing id. A cursor column holding future timestamps parks the bridge until real time catches up.

Deletes are not crossing the bridge

Expected on a watch bridge. Watch polls for rows that exist, so it sees inserts, and updates when the cursor is a timestamp, but a deleted row is simply absent from the next poll and indistinguishable from one that was never there. Deletes need a CDC trigger, which reads them from the change log.

On a CDC bridge, check that delete is among its operations, and that the target has key columns — an append-only (insert mode) target records rows and has nothing to delete by, so it never applies deletes. A TRUNCATE is not a delete of every row: it is its own event, PostgreSQL-only and off by default. When the source is truncated the timeline says so in an amber entry, and the destination keeps its rows unless the bridge captures truncate — see operations.

A connection will not test

An error beginning SSH: comes from the tunnel, not the database — the jump host refused the key, the user, or the forward. Check the SSH credentials on their own before looking at the database.SSH: the host key of … has CHANGED means the jump host presented a different key than the one recorded for this connection: do not just clear the field — confirm the new fingerprint with whoever runs the host first.

An error beginning TLS:, or mentioning a certificate, altname or self-signed, is the verification you asked for doing its job. Self-signed certificate in chain means the server's CA is not one Syncle trusts: paste it into the connection's CA certificate field. Hostname/IP does not match certificate's altnames means the certificate was issued for a different name than the one in the host field: dial the name on the certificate, or set the expected server name. Dropping to Encrypt only makes the error go away by no longer checking — use it to confirm the diagnosis, not as the fix.

Without a tunnel, the usual causes are the database not listening on an interface Syncle can reach, or TLS. Syncle connects out to your database, so the database has to accept a connection from the Docker host — and it has to be listening on an address that is not only the host's own loopback (PostgreSQL's listen_addresses, MySQL's bind-address), with a pg_hba.conf rule or grant that admits the Docker network.

The address itself needs nothing from you: a database on the machine running Syncle is reached by typing localhost, as you would expect. Inside a container that name means the container, so Syncle reads it as the host and dials there instead — host.docker.internal where Docker provides that name, and the container's gateway otherwise. It is applied on the way to the database only: the connection still shows the address you typed. 127.0.0.1 and a loopback address inside a connection string are treated the same way, and SYNCLE_HOST_GATEWAY overrides the address if your Docker reaches the host at some other one.

One place this does not apply, because there it would be wrong: a connection that goes through an SSH tunnel. Its host is resolved on the far side of the bastion, where localhost means the bastion's own database — which is usually exactly what a tunnel is for.

Rows are failing rather than syncing

Click a failed delivery on the timeline: it shows the row that was sent and the error that came back. Failed rows can be retried in place without rerunning the whole job.

A bridge that fails everything usually has a destination mismatch — key columns that are not unique in the target, or a type the target will not accept. A bridge that fails intermittently against an HTTP endpoint is usually being rate limited; raise the minimum delay between requests, or lower the batch size, in the bridge's delivery settings.

A live bridge paused itself

A watch or CDC bridge that stops on its own always leaves the reason on the job, and always stops without moving its cursor — so nothing was skipped, and starting it again picks up the same rows. What the message says tells you what to fix first:

The job saysWhat happenedWhat to do
Paused after a failed delivery (onError=abort)the bridge is set to stop at the first failure, and it didfix what the error names, then start the bridge. If a single bad row should not stop everything, switch On failure to setting failed rows aside
this is not a few bad rowsa batch failed, and splitting it found nothing that would go through — the destination is rejecting everythingcheck the destination is reachable, the table exists, and its columns still match the source
N batches in a row delivered nothingthe same, seen across several small batchesas above. Rows from the earlier batches are in the dead-letter queue — retry them after starting the bridge
the dead-letter queue is fullfailed rows reached SYNCLE_DEAD_LETTER_MAX_ROWSretry or discard the queue, then start the bridge. A queue that keeps filling means the cause was never fixed
progress could not be savedSyncle's own database was unreachable. The job is marked failed rather than pausedbring the metadata store back, then start the bridge

A bridge says it cannot resume where it stopped

A live bridge resumes from a position in the source's change log, and that position no longer exists. The message names the cause: the PostgreSQL replication slot was dropped or invalidated (the server hit max_slot_wal_keep_size, or Syncle's own SYNCLE_SLOT_MAX_BYTES guard gave it up, or someone dropped it by hand); MySQL purged the binlog file; MongoDB's oplog rolled past the resume token; or a MySQL connection now reaches a different server than the one that issued the position.

What changed at the source in between cannot be read any more, so there are two steps: start the bridge and confirm Continue from now, then run a replay of the same bridge to bring the destination up to date (writes are upserts, so replaying over existing rows is safe). To stop it recurring, keep the log for longer than a bridge is ever paused: max_slot_wal_keep_size, binlog_expire_logs_seconds, the oplog size.

The source database's disk is filling up

On PostgreSQL, look for a replication slot nothing is reading:

psql

SELECT slot_name, active,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS pinned
FROM pg_replication_slots ORDER BY 3 DESC;

A syncle_slot_… that is not active belongs to a bridge that is paused or failed; the bridge's job view shows the same figure with a warning. Start the bridge and it catches up and lets the WAL go; delete the bridge and the slot goes with it. If the bridge is already gone — it was deleted while the server was unreachable — Syncle keeps retrying the drop (GET /api/bridges/cdc/cleanups), or drop it yourself with SELECT pg_drop_replication_slot('syncle_slot_…');. Then set max_slot_wal_keep_size so that it cannot happen again; the details are under replication slots.

Rows will not leave the dead-letter queue

A retry that still fails keeps the entry and replaces its error with the new one, so the entry always shows the latest reason. Usually the destination still refuses the row — fix the constraint, type or mapping it names and retry again.

An entry that a retry leaves untouched, offering Write recorded row, is a different case: its source row no longer exists and the bridge does not propagate deletes, so Syncle cannot tell whether the row it recorded is still the newest version. Write it anyway if the destination is meant to keep rows the source has dropped; discard it otherwise. How bridges work has the reasoning.

Still stuck

syncle logs api carries the server side of anything the interface could not explain. If it looks like a bug, open an issue with the engine, the trigger type, and that log; if you are unsure whether it is a bug, Discussions is the better place to start.