Keep multiple databases in sync without kafka

Syncle connects to PostgreSQL, MySQL, SQLite, MongoDB and Redis. You pick a table on one of them and where it should go on another. Syncle copies the rows across, then keeps copying as the source changes.

curl -fsSL https://syncle.dev/install | sh -s -- up

Docker with Compose v2 is all you need. Runs on your own machine, MIT licensed.

What it does

One Postgres table feeding three other databases at the same time.

A bridge connects one source to one or more destinations. Any engine can sit on either side, so a Postgres table can go to MongoDB, to Redis, or to another Postgres. A destination can also be an HTTP endpoint, if you are feeding a service instead of a database.

If the destination table does not exist yet, Syncle creates it and works out the column types for that engine. Writes are upserts, so running a sync twice will not leave you with two copies of a row. A job that dies partway through restarts from the last row it recorded, not from the beginning.

Three ways to run a bridge

Replay reads the whole table once and stops. Use it to backfill or to migrate.
Watch checks for new rows against a cursor you choose, such as an id or an updated_at column. It works on all five engines.
CDC reads the database's own change log. Nothing polls, and a change shows up as soon as it commits.

Two things to know. SQLite has no change log, so watch is the only option there. Redis keyspace notifications are not durable, so a Redis CDC bridge misses anything that changes while Syncle is off. Both are covered in how bridges work and CDC setup.

Installing

Run the command above. It checks for Docker, pulls the image, starts four containers and opens localhost:3002.
Create your account. The setup form opens with a one-time token already in it.
Build a bridge. Choose a source table, choose a destination, press start.

After that you use the launcher:

syncle up — start it and open the interface
syncle down — stop it, keeping your data
syncle logs — follow what the bridges are doing
syncle update — move to the newest release
syncle uninstall — remove everything, data included

The script is install.sh if you want to read it before you run it. The installation page covers the manual Docker Compose route.

Documentation

Overview — What Syncle is, the pieces it is made of, and where to start reading.
Installation — The one-command install, the syncle launcher, the manual Docker Compose route, ports, and updating or removing an install.
Quickstart — From syncle up to a first working bridge: the setup token, connecting a database, and reading the delivery timeline.
How bridges work — Bridges, jobs and deliveries; the three trigger modes; delivery guarantees; destinations; and the tuning knobs.
CDC setup — Per-engine prerequisites for real-time change data capture, what Syncle provisions for you, and where each engine falls short.
Database workbench — Browsing and editing tables, the query editor, schema and ER diagrams, DDL, backup and restore, and SSH tunnels.
Configuration — Every environment variable with its default, the in-app settings, and where the config files live.
HTTP API — The REST API under /api: authentication, response envelopes, and every endpoint for connections, bridges and jobs.
Troubleshooting — The failures that come up most: CDC that never fires, watch bridges that deliver nothing, connections that will not test, and rows that fail.
Self-hosting & security — Running Syncle beyond localhost: accounts and sessions, encryption, what to back up, and a troubleshooting list.

Questions and bugs

Setup questions go in Discussions. Bugs go in the issue tracker, including anywhere the docs and the software disagree. For a security problem, email instead of opening an issue.