Barrel
FR GitHub

barrel_crypto

Encryption at rest for Barrel: the ciphers, key derivation and key providers behind the encryption option of a database.

What it solves

A database per tenant or per agent should be unreadable without its key, on disk and in backups. barrel_crypto derives a key per database from a master secret (or asks your key provider), and the document store, vector store, disk BM25 files and blob attachments are encrypted with it.

When to use it

  • Each tenant, agent or space needs its own key.
  • Processes in one Erlang node must not read each other's spaces: encryption is that boundary.
  • You keep keys in a KMS: write a key provider module.

When not to

  • You store attachments in S3 through barrel_att_s3: those bytes are not encrypted.
  • You need key rotation or to encrypt an existing database in place: neither exists.

How it fits into Barrel

Builds on
No other Barrel library
Other dependencies
OTP crypto
On its own
Yes, it has no dependencies beyond OTP crypto.

Install

rebar.config deps
{barrel_crypto, "~> 1.0"}

Configuration

Keys

KeyDefaultWhat it does
BARREL_ENCRYPTION_KEY none Master secret: 32 raw bytes, 64 hex characters, or a passphrase
{barrel_crypto, encryption_key} none Used when the environment variable is not set
BARREL_ENCRYPTION_SALT / encryption_salt barrel_default_salt_v1 HKDF salt

encryption option

KeyDefaultWhat it does
disabled default Plaintext
default Per-database key derived from the master secret
#{provider => Mod} Your module returns the 32-byte key for a keyspace

The most used keys only. For every option, see the configuration reference and the API reference.

The API by task

Keys

TaskCallReturns
Key for a database barrel_keyprovider:key_for_db/2 {ok, Key}, {ok, plaintext}, or {error, _}
Your own provider callback key_for_db(Keyspace) {ok, <<_:256>>}
Derive barrel_crypto:derive_key/1,2 a key

Ciphers

TaskCallReturns
Seal, open (AES-256-GCM) barrel_crypto:encrypt/2, decrypt/2 envelope, plaintext or {error, decryption_failed}
Terms barrel_crypto:encrypt_term/2, decrypt_term/2 envelope, term
File slices (AES-256-CTR) barrel_crypto:ctr_crypt/3,4, new_nonce/1 bytes
Check a key barrel_crypto:key_check_new/1, key_check_verify/2 token, boolean

An encrypted database, checked on disk

From a verified run: documents, a vector, disk BM25 and a blob attachment, then a search of the files for the plaintext.

Erlang
os:putenv("BARREL_ENCRYPTION_KEY", "a long passphrase"),
{ok, _} = application:ensure_all_started(barrel),
Opts = fun(Enc) -> #{encryption => Enc,
    docdb => #{data_dir => Dir},
    vectordb => #{dimension => 3, bm25_backend => disk,
                  db_path => Dir ++ "/secret_vec"}} end,
{ok, Db} = barrel:open(secret, Opts(default)),
{ok, _} = barrel:put_doc(Db, #{<<"id">> => <<"a">>,
    <<"note">> => <<"TOPSECRETMARKER">>}),
ok = barrel:vector_add(Db, <<"a">>, <<"TOPSECRETVECTEXT">>,
                       #{}, [0.1, 0.2, 0.3]),
{ok, _} = barrel:put_attachment(Db, <<"a">>, <<"f.txt">>,
                                <<"TOPSECRETBLOB">>),
ok = barrel:close(Db),
io:format("files containing plaintext marker: ~s",
    [os:cmd("grep -rl -a 'TOPSECRET' " ++ Dir
            ++ " | wc -l")]),
io:format("plaintext open: ~p~n",
          [barrel:open(secret, Opts(disabled))]),
os:putenv("BARREL_ENCRYPTION_KEY", "another passphrase"),
io:format("wrong key open: ~p~n",
          [barrel:open(secret, Opts(default))]).
Output, from a run on barrel_crypto 1.0.0 (the same write without encryption left the marker in 1 file)
files containing plaintext marker:        0
plaintext open: {error,db_is_encrypted}
wrong key open: {error,wrong_encryption_key}

Limits to know

  • No key rotation and no in-place encryption of an existing database ({error, cannot_encrypt_existing_db}).
  • Keys are never written to disk, but a cleartext key-check file CRYPTO sits in the database directory.
  • CTR mode is unauthenticated, and keys are not zeroized in memory (both documented).
  • decrypt/2 passes through input that does not start with BR; a short plaintext starting with BR crashes it.
Known gap

Attachments stored in S3 through barrel_att_s3 are not encrypted, even when the database is (verified: the object in the bucket held the plaintext).

Up and down to moveEnter to open