Granite Embedding Small English R2 vs jina-embeddings-v5-omni-small

Granite Embedding Small English R2

A tiny English embedding cousin for retrieval over long documents and code, 47M parameters with 384-dimension vectors and an 8,192-token input, under Apache 2.0 with a permissive data story. It runs on any hardware through Sentence Transformers, vLLM or llama.cpp. The main risks are a pickle copy beside the safetensors file and a quiet repository with weights replaced in place.

jina-embeddings-v5-omni-small

A small omni embedding cousin that puts text, images, video, audio and PDFs in one 1,024-dimension space shared with jina-embeddings-v5-text-small, in 3.4 GB of BF16 files or a 0.4 GB GGUF. It's for non-commercial use only under CC BY-NC 4.0, and every way to run it executes Jina's own code or a llama.cpp fork rather than a released runtime's.

Granite Embedding Small English R2

1. Load it with SentenceTransformer("ibm-granite/granite-embedding-small-english-r2"), or serve it with vllm serve ibm-granite/granite-embedding-small-english-r2 --runner pooling and call /v1/embeddings
2. Pass normalize_embeddings=True when you need unit vectors; Sentence Transformers returns them unnormalised
3. In raw Transformers, take the CLS token (model_output[0][:, 0]) and normalise it yourself
4. Don't add query or passage prefixes; the model takes plain text
5. Load the safetensors file and ignore pytorch_model.bin, and chunk text past 8,192 tokens
6. Treat retrieved text as untrusted before passing it to a model

jina-embeddings-v5-omni-small

1. Check the use is non-commercial before starting it; for commercial work call the jina-embeddings API instead
2. Start it with pip install vllm==0.20.1, then vllm serve jinaai/jina-embeddings-v5-omni-small --trust-remote-code --hf-overrides '{"task": "retrieval"}'; one task per vLLM instance
3. Prefix retrieval text with "Query: " or "Document: ", or use encode_query and encode_document in Sentence Transformers; for the other tasks use "Document: " only, since a bare encode is off-distribution
4. Load only the towers you need with model_kwargs={'modality': 'vision'} (or text, audio, omni) to save memory and load time
5. Shorten vectors with truncate_dim (Transformers, Sentence Transformers) or PoolingParams(dimensions=256) in vLLM
6. Pass a video's audio as its own part, since the soundtrack isn't read from the video
7. Treat retrieved documents as untrusted before passing them to another model