barrel_att_s3
An attachment backend for barrel_docdb that keeps file bytes in an S3-compatible bucket instead of the local RocksDB blob store.
What it solves
Large files fill local disks and are hard to share between nodes. With barrel_att_s3 the document stays in barrel_docdb while its attachments go to object storage, with the attachment feed, replication, branching (server-side copy), multipart uploads and cleanup of abandoned uploads.
When to use it
- Attachments are large or many, and local disk is the limit.
- You already run S3-compatible storage (RustFS and Garage are tested in CI).
When not to
- The database is encrypted and the files must be too: S3 attachments are stored in plaintext.
- You need conditional writes on Garage: it cannot enforce them.
How it fits into Barrel
- Builds on
barrel_docdb- Used by
- No other Barrel library depends on it
- Optional in
barrel_docdb- Other dependencies
- livery_s3, livery, rocksdb, hlc, mimerl
- On its own
- No. It is a backend that barrel_docdb loads through
att_opts.
Install
{barrel_att_s3, "~> 0.1"} Not in the default umbrella build: use rebar3 as s3 compile.
Configuration
att_opts
| Key | Default | What it does |
|---|---|---|
backend | blob | Set s3 |
s3.bucket | required | Must exist; missing gives {att_store_open_failed, missing_bucket} |
s3.endpoint, s3.region | none | Where the store is |
s3.access_key_id, s3.secret_access_key | none | Credentials |
s3.part_size | 8 MiB | Multipart part size |
App env (barrel_att_s3)
| Key | Default | What it does |
|---|---|---|
multipart_gc_interval | 1 h | Cleanup of abandoned uploads; 0 disables |
multipart_gc_max_age | 24 h | Age before an upload is abandoned |
The most used keys only. For every option, see the configuration reference.
The API by task
Through barrel_docdb
| Task | Call | Returns |
|---|---|---|
| Use S3 for a database | barrel_docdb:create_db/2 with att_opts | {ok, Pid} |
| Store, read, delete | put_attachment/4,5, get_attachment/3, delete_attachment/3 | {ok, Info}, {ok, Bytes}, ok |
| Write only if absent | put_attachment/5 with create_only => true | {error, {conflict, Info}} if it exists |
| Write if unchanged | put_attachment/5 with expected_etag | {error, conditional_writes_unsupported} on Garage |
| Stream | open_attachment_writer/4,5, open_attachment_stream/3,4 | writer and reader handles |
Attachments in a local RustFS bucket
From a verified run against RustFS 1.0.0 in a container. Objects are keyed <db>/<hex doc id>/<name>.
S3 = #{bucket => <<"barrel-matrix">>,
endpoint => <<"http://127.0.0.1:29000">>,
region => <<"us-east-1">>,
access_key_id => <<"s3testadmin">>,
secret_access_key => <<"s3testsecret">>},
{ok, _} = application:ensure_all_started(barrel_att_s3),
{ok, _} = barrel_docdb:create_db(<<"s3db">>,
#{data_dir => Dir,
att_opts => #{backend => s3, s3 => S3}}),
{ok, _} = barrel_docdb:put_doc(<<"s3db">>,
#{<<"id">> => <<"doc1">>}),
{ok, Info} = barrel_docdb:put_attachment(<<"s3db">>,
<<"doc1">>, <<"greeting.txt">>, <<"Hello, World!">>),
io:format("put: ~p~n", [Info]),
io:format("get: ~p~n", [barrel_docdb:get_attachment(
<<"s3db">>, <<"doc1">>, <<"greeting.txt">>)]),
io:format("create_only again: ~p~n",
[element(1, barrel_docdb:put_attachment(<<"s3db">>,
<<"doc1">>, <<"greeting.txt">>, <<"second">>,
#{create_only => true}))]). put: #{name => <<"greeting.txt">>,length => 13,
digest => <<"sha256-dffd6021...a362182986f">>,
content_type => <<"text/plain">>,chunked => false}
get: {ok,<<"Hello, World!">>}
create_only again: error Limits to know
- Branching copies each attachment in the background; until done, reads answer
{error, {att_sync_pending, _}}. - Deleting a closed database needs
att_optsagain. Re-creating an open database with newatt_optssilently does nothing. - MinIO and AWS S3 are documented as supported but not tested in CI.
- Feed metadata stays local in
<db>/attachments/feed.db.
Attachments are not encrypted by database encryption: with encryption => default, the object in the bucket held the plaintext.