Write on every node
Run a copy of the database on every site, device or region. Each copy takes writes while its link is down, and replication brings every copy back to the same state without losing an edit.
Active-active: no leader, no primary. Built from barrel_docdb replication tasks and the _sync wire served by barrel_server.
The problem
Sites, devices and regions each need their data locally, plus a central copy, over links that come and go. A primary database makes every site wait on the network, and last-write-wins sync quietly drops one side of every concurrent edit.
How it fits together
-
Site A (your Erlang release)
- Your application reads and writes locally
- inventory barrel_docdb, on local disk
- continuous task, both ways
-
Hub (barrel_server)
- /db/inventory/_sync the replication wire
- inventory the central copy
- continuous task, both ways
-
Site B and others
- Your application reads and writes locally
- inventory its own copy
Every copy accepts writes. Each node runs one task against the hub; the hub passes changes on to the other nodes. Filters and channels decide which documents a node receives.
Two sites edit the same task
Cut the link, edit on both sides, sync. Neither edit is lost: both copies pick the same winner and keep the other edit as a conflict until someone resolves it.
The call behind your last stepDemonstration in your browser, following barrel's rules
Build it
Five steps: a hub, then the same few calls on every node.
-
Run a hub
One barrel_server with a token the nodes share. Replication does not create databases, so create it first.
%% sys.config of the hub [{barrel_server, [ {http_port, 8080}, {auth, #{tokens => [<<"sync-token">>]}} ]}].curl -X PUT hub.example:8080/db/inventory \ -H 'authorization: Bearer sync-token' -
Embed Barrel on each node
Create the same database locally. Give the token by origin in the app env, so task configs stay free of secrets.
{ok, _} = application:ensure_all_started(barrel_docdb), {ok, _} = barrel_docdb:create_db(<<"inventory">>), application:set_env(barrel_docdb, sync_auth, #{<<"http://hub.example:8080">> => <<"sync-token">>}). -
Start a continuous task in both directions
Tasks persist across restarts. A local source wakes the task on each change; the remote side is polled, 500 ms after data and backing off to 15 s when idle.
{ok, TaskId} = barrel_rep_tasks:start_task(#{ source => <<"inventory">>, target => <<"http://hub.example:8080/db/inventory">>, mode => continuous, direction => both}). -
Write locally, linked or not
Writes never wait for the hub. When the link drops, the task backs off from 1 s to 60 s, records
last_errorand staysrunning; it catches up from its checkpoint when the link returns.{ok, _} = barrel_docdb:put_doc(<<"inventory">>, #{<<"id">> => <<"sku:42">>, <<"qty">> => 7}), {ok, #{status := running}} = barrel_rep_tasks:get_task(TaskId). -
Resolve the conflicts you care about
Two sites that edit the same document while apart both keep their edit: every copy picks the same winner and keeps the other as a conflict. Resolve it by writing a version that replaces the versions you observed. A
conflict_resolvergiven atcreate_dbcan do this for you.{ok, #{<<"_observed">> := Seen}} = barrel_docdb:get_doc(<<"inventory">>, <<"sku:42">>, #{conflicts => true}), {ok, _} = barrel_docdb:resolve_conflict(<<"inventory">>, <<"sku:42">>, Seen, {merge, #{<<"qty">> => 5}}).{ok,#{id => <<"task:1">>, rev => <<"000001a1b879d59500000000@w7bcb971434496877">>, conflicts_resolved => 1}}
What Barrel does
- Orders writes with a hybrid logical clock and version vectors
- Picks the same winner on every copy, and keeps concurrent edits as conflicts
- Checkpoints each batch and resumes after a failure
- Moves attachments with their documents, content-addressed
- Filters at the source by path, query or channel
What your application does
- Chooses the topology: one hub, several, or a mesh
- Decides which data goes where, with filters or channels
- Hands each node its token through
sync_auth - Resolves conflicts, or sets a
conflict_resolver - Watches task status and
last_error
Limits
- Vector indexes do not replicate. In record mode the documents travel and each node rebuilds its index.
- Provenance (actor, session, source) stays on the node that took the write; arrivals are marked replicated.
- A document that leaves a channel stays on replicas that already have it.
- The
barrelfacade does not expose replication: callbarrel_rep_taskson the docdb name. - A peer whose clock is too far ahead is refused with
clock_skew. HTTP/3 listeners do not check client certificates yet.