Embedding Providers — 9 Options

Beluga AI v2 provides a unified embedding.Embedder interface for converting text into dense vector representations. All providers register via init() and are instantiated through the global registry.

The unified interface means your RAG pipeline, vector store integration, and retrieval logic work identically regardless of which embedding provider you choose. Switch providers by changing a single import and configuration value.

Interface

type Embedder interface {
    Embed(ctx context.Context, texts []string) ([][]float32, error)
    EmbedSingle(ctx context.Context, text string) ([]float32, error)
    Dimensions() int
}

Registry Usage

import (
    "github.com/lookatitude/beluga-ai/config"
    "github.com/lookatitude/beluga-ai/rag/embedding"

    // Register the provider you need via blank import
    _ "github.com/lookatitude/beluga-ai/rag/embedding/providers/openai"
)

func main() {
    emb, err := embedding.New("openai", config.ProviderConfig{
        APIKey: os.Getenv("OPENAI_API_KEY"),
    })
    if err != nil {
        log.Fatal(err)
    }

    vectors, err := emb.Embed(context.Background(), []string{"hello world"})
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Dimensions: %d\n", emb.Dimensions())
}

Available Providers

Provider	Registry Name	Default Model	Default Dimensions
OpenAI	openai	text-embedding-3-small	1536
Cohere	cohere	embed-english-v3.0	1024
Google	google	text-embedding-004	768
Ollama	ollama	nomic-embed-text	768
Jina	jina	jina-embeddings-v2-base-en	768
Voyage	voyage	voyage-2	1024
Mistral	mistral	mistral-embed	1024
Sentence Transformers	sentence_transformers	all-MiniLM-L6-v2	384
In-Memory	inmemory	N/A (hash-based)	128

Provider Discovery

List all registered providers at runtime:

names := embedding.List()
// Returns sorted list: ["cohere", "google", "inmemory", "jina", ...]

Choosing a Provider

Use Case	Recommended Provider
General-purpose, best cost/quality ratio	OpenAI
Asymmetric search (separate doc/query embeddings)	Cohere, Voyage
Code retrieval	Voyage (voyage-code-2)
Multilingual (100+ languages)	Cohere (embed-multilingual-v3.0)
Local/offline development	Ollama
Testing and unit tests	In-Memory
Self-hosted with open-source models	Sentence Transformers
Google Cloud integration	Google

Middleware

All embedders support middleware for cross-cutting concerns such as logging, caching, and tracing:

emb := embedding.ApplyMiddleware(baseEmb,
    loggingMiddleware,
    cachingMiddleware,
)

Hooks

Hooks allow observing embedding operations without wrapping the interface:

emb = embedding.ApplyMiddleware(baseEmb,
    embedding.WithHooks(embedding.Hooks{
        BeforeEmbed: func(ctx context.Context, texts []string) error {
            log.Printf("Embedding %d texts", len(texts))
            return nil
        },
        AfterEmbed: func(ctx context.Context, embeddings [][]float32, err error) {
            if err != nil {
                log.Printf("Embedding failed: %v", err)
            }
        },
    }),
)