Barrel
FR GitHub

Write on every node

Run a copy of the database on every site, device or region. Each copy takes writes while its link is down, and replication brings every copy back to the same state without losing an edit.

Active-active: no leader, no primary. Built from barrel_docdb replication tasks and the _sync wire served by barrel_server.

The problem

Sites, devices and regions each need their data locally, plus a central copy, over links that come and go. A primary database makes every site wait on the network, and last-write-wins sync quietly drops one side of every concurrent edit.

How it fits together

  1. Site A (your Erlang release)

    • Your application reads and writes locally
    • inventory barrel_docdb, on local disk
  2. continuous task, both ways
  3. Hub (barrel_server)

    • /db/inventory/_sync the replication wire
    • inventory the central copy
  4. continuous task, both ways
  5. Site B and others

    • Your application reads and writes locally
    • inventory its own copy

Every copy accepts writes. Each node runs one task against the hub; the hub passes changes on to the other nodes. Filters and channels decide which documents a node receives.

Two sites edit the same task

Cut the link, edit on both sides, sync. Neither edit is lost: both copies pick the same winner and keep the other edit as a conflict until someone resolves it.

The call behind your last stepDemonstration in your browser, following barrel's rules

 

Build it

Five steps: a hub, then the same few calls on every node.

  1. Run a hub

    One barrel_server with a token the nodes share. Replication does not create databases, so create it first.

    %% sys.config of the hub
    [{barrel_server, [
       {http_port, 8080},
       {auth, #{tokens => [<<"sync-token">>]}}
     ]}].
  2. Embed Barrel on each node

    Create the same database locally. Give the token by origin in the app env, so task configs stay free of secrets.

    Erlang, on each node
    {ok, _} = application:ensure_all_started(barrel_docdb),
    {ok, _} = barrel_docdb:create_db(<<"inventory">>),
    application:set_env(barrel_docdb, sync_auth,
        #{<<"http://hub.example:8080">> => <<"sync-token">>}).
  3. Start a continuous task in both directions

    Tasks persist across restarts. A local source wakes the task on each change; the remote side is polled, 500 ms after data and backing off to 15 s when idle.

    Erlang, on each node
    {ok, TaskId} = barrel_rep_tasks:start_task(#{
        source    => <<"inventory">>,
        target    => <<"http://hub.example:8080/db/inventory">>,
        mode      => continuous,
        direction => both}).
  4. Write locally, linked or not

    Writes never wait for the hub. When the link drops, the task backs off from 1 s to 60 s, records last_error and stays running; it catches up from its checkpoint when the link returns.

    Erlang
    {ok, _} = barrel_docdb:put_doc(<<"inventory">>,
        #{<<"id">> => <<"sku:42">>, <<"qty">> => 7}),
    {ok, #{status := running}} =
        barrel_rep_tasks:get_task(TaskId).
  5. Resolve the conflicts you care about

    Two sites that edit the same document while apart both keep their edit: every copy picks the same winner and keeps the other as a conflict. Resolve it by writing a version that replaces the versions you observed. A conflict_resolver given at create_db can do this for you.

    Erlang
    {ok, #{<<"_observed">> := Seen}} =
        barrel_docdb:get_doc(<<"inventory">>, <<"sku:42">>,
                             #{conflicts => true}),
    {ok, _} = barrel_docdb:resolve_conflict(<<"inventory">>,
        <<"sku:42">>, Seen, {merge, #{<<"qty">> => 5}}).
    Result shape, from a recorded run (other names)
    {ok,#{id => <<"task:1">>,
          rev => <<"000001a1b879d59500000000@w7bcb971434496877">>,
          conflicts_resolved => 1}}

What Barrel does

  • Orders writes with a hybrid logical clock and version vectors
  • Picks the same winner on every copy, and keeps concurrent edits as conflicts
  • Checkpoints each batch and resumes after a failure
  • Moves attachments with their documents, content-addressed
  • Filters at the source by path, query or channel

What your application does

  • Chooses the topology: one hub, several, or a mesh
  • Decides which data goes where, with filters or channels
  • Hands each node its token through sync_auth
  • Resolves conflicts, or sets a conflict_resolver
  • Watches task status and last_error

Limits

  • Vector indexes do not replicate. In record mode the documents travel and each node rebuilds its index.
  • Provenance (actor, session, source) stays on the node that took the write; arrivals are marked replicated.
  • A document that leaves a channel stays on replicas that already have it.
  • The barrel facade does not expose replication: call barrel_rep_tasks on the docdb name.
  • A peer whose clock is too far ahead is refused with clock_skew. HTTP/3 listeners do not check client certificates yet.

Go further

Up and down to moveEnter to open