Barrel
FR GitHub

barrel_att_s3

An attachment backend for barrel_docdb that keeps file bytes in an S3-compatible bucket instead of the local RocksDB blob store.

What it solves

Large files fill local disks and are hard to share between nodes. With barrel_att_s3 the document stays in barrel_docdb while its attachments go to object storage, with the attachment feed, replication, branching (server-side copy), multipart uploads and cleanup of abandoned uploads.

When to use it

  • Attachments are large or many, and local disk is the limit.
  • You already run S3-compatible storage (RustFS and Garage are tested in CI).

When not to

  • The database is encrypted and the files must be too: S3 attachments are stored in plaintext.
  • You need conditional writes on Garage: it cannot enforce them.

How it fits into Barrel

Builds on
barrel_docdb
Used by
No other Barrel library depends on it
Optional in
barrel_docdb
Other dependencies
livery_s3, livery, rocksdb, hlc, mimerl
On its own
No. It is a backend that barrel_docdb loads through att_opts.

Install

rebar.config deps
{barrel_att_s3, "~> 0.1"}

Not in the default umbrella build: use rebar3 as s3 compile.

Configuration

att_opts

KeyDefaultWhat it does
backend blob Set s3
s3.bucket required Must exist; missing gives {att_store_open_failed, missing_bucket}
s3.endpoint, s3.region none Where the store is
s3.access_key_id, s3.secret_access_key none Credentials
s3.part_size 8 MiB Multipart part size

App env (barrel_att_s3)

KeyDefaultWhat it does
multipart_gc_interval 1 h Cleanup of abandoned uploads; 0 disables
multipart_gc_max_age 24 h Age before an upload is abandoned

The most used keys only. For every option, see the configuration reference.

The API by task

Through barrel_docdb

TaskCallReturns
Use S3 for a database barrel_docdb:create_db/2 with att_opts {ok, Pid}
Store, read, delete put_attachment/4,5, get_attachment/3, delete_attachment/3 {ok, Info}, {ok, Bytes}, ok
Write only if absent put_attachment/5 with create_only => true {error, {conflict, Info}} if it exists
Write if unchanged put_attachment/5 with expected_etag {error, conditional_writes_unsupported} on Garage
Stream open_attachment_writer/4,5, open_attachment_stream/3,4 writer and reader handles

Attachments in a local RustFS bucket

From a verified run against RustFS 1.0.0 in a container. Objects are keyed <db>/<hex doc id>/<name>.

Erlang
S3 = #{bucket => <<"barrel-matrix">>,
       endpoint => <<"http://127.0.0.1:29000">>,
       region => <<"us-east-1">>,
       access_key_id => <<"s3testadmin">>,
       secret_access_key => <<"s3testsecret">>},
{ok, _} = application:ensure_all_started(barrel_att_s3),
{ok, _} = barrel_docdb:create_db(<<"s3db">>,
    #{data_dir => Dir,
      att_opts => #{backend => s3, s3 => S3}}),
{ok, _} = barrel_docdb:put_doc(<<"s3db">>,
                               #{<<"id">> => <<"doc1">>}),
{ok, Info} = barrel_docdb:put_attachment(<<"s3db">>,
    <<"doc1">>, <<"greeting.txt">>, <<"Hello, World!">>),
io:format("put: ~p~n", [Info]),
io:format("get: ~p~n", [barrel_docdb:get_attachment(
    <<"s3db">>, <<"doc1">>, <<"greeting.txt">>)]),
io:format("create_only again: ~p~n",
    [element(1, barrel_docdb:put_attachment(<<"s3db">>,
        <<"doc1">>, <<"greeting.txt">>, <<"second">>,
        #{create_only => true}))]).
Output, from a run on barrel_att_s3 0.1.4 (digest trimmed)
put: #{name => <<"greeting.txt">>,length => 13,
       digest => <<"sha256-dffd6021...a362182986f">>,
       content_type => <<"text/plain">>,chunked => false}
get: {ok,<<"Hello, World!">>}
create_only again: error

Limits to know

  • Branching copies each attachment in the background; until done, reads answer {error, {att_sync_pending, _}}.
  • Deleting a closed database needs att_opts again. Re-creating an open database with new att_opts silently does nothing.
  • MinIO and AWS S3 are documented as supported but not tested in CI.
  • Feed metadata stays local in <db>/attachments/feed.db.
Known gap

Attachments are not encrypted by database encryption: with encryption => default, the object in the bucket held the plaintext.

Up and down to moveEnter to open