barrel_embed_colbert (barrel_embed v2.3.0)
View SourceColBERT late interaction embedding provider
Uses ColBERT models for multi-vector embeddings. Each document produces multiple vectors (one per token) for fine-grained matching.
Dependencies (transformers, torch) are installed automatically in the managed venv on first use.
Configuration
Config = #{
model => "colbert-ir/colbertv2.0", %% Model name (default, 128 dims)
python => "python3", %% Python executable (default)
timeout => 120000 %% Timeout in ms (default)
}.Multi-Vector Format
Unlike single-vector embeddings, ColBERT produces a list of vectors:
[[0.1, 0.2, ...], [0.3, 0.4, ...], ...] %% One vector per tokenLate Interaction
ColBERT scoring uses MaxSim:
Score(Q, D) = sum(max(qi · dj for all dj in D) for all qi in Q)This enables fine-grained token-level matching.
Supported Models
- "colbert-ir/colbertv2.0" - Default, 128 dimensions - "answerdotai/answerai-colbert-small-v1" - 96 dimensions, smaller - "jinaai/jina-colbert-v2" - 128 dimensions, long context (8192 tokens)
Use Cases
- Fine-grained semantic matching - Passage retrieval with token-level scoring - Question answering
Summary
Functions
Check if provider is available.
Get dimension for this provider.
Generate single-vector embedding (mean pooling of token vectors). Note: For ColBERT, use embed_multi/2 to get full multi-vector output.
Generate single-vector embeddings for batch (mean pooling).
Generate multi-vector embeddings for multiple texts.
Generate multi-vector embedding for a single text. Returns a list of token vectors.
Initialize the provider.
Calculate MaxSim score between query and document multi-vectors. This is the standard ColBERT scoring function. Score = sum(max(qi · dj for all dj in D) for all qi in Q)
Provider name.
Types
-type multi_vector() :: [[float()]].
Functions
Check if provider is available.
-spec dimension(map()) -> pos_integer().
Get dimension for this provider.
Generate single-vector embedding (mean pooling of token vectors). Note: For ColBERT, use embed_multi/2 to get full multi-vector output.
Generate single-vector embeddings for batch (mean pooling).
-spec embed_batch_multi([binary()], map()) -> {ok, [multi_vector()]} | {error, term()}.
Generate multi-vector embeddings for multiple texts.
-spec embed_multi(binary(), map()) -> {ok, multi_vector()} | {error, term()}.
Generate multi-vector embedding for a single text. Returns a list of token vectors.
Initialize the provider.
-spec maxsim_score(multi_vector(), multi_vector()) -> float().
Calculate MaxSim score between query and document multi-vectors. This is the standard ColBERT scoring function. Score = sum(max(qi · dj for all dj in D) for all qi in Q)
-spec name() -> atom().
Provider name.