jina-embeddings-v5-omni-small vs Qwen3-Embedding-4B

jina-embeddings-v5-omni-small

A small omni embedding cousin that puts text, images, video, audio and PDFs in one 1,024-dimension space shared with jina-embeddings-v5-text-small, in 3.4 GB of BF16 files or a 0.4 GB GGUF. It's for non-commercial use only under CC BY-NC 4.0, and every way to run it executes Jina's own code or a llama.cpp fork rather than a released runtime's.

Qwen3-Embedding-4B

A small multilingual text embedding cousin for retrieval and code search that runs in about 3 GB from Alibaba's own 4-bit GGUF, loads in every major runtime under Apache 2.0 and has two hosts, DeepInfra and SiliconFlow. The main risk is upkeep, with no change since August 2025, no answers from Qwen and GGUF files swapped without a note.

jina-embeddings-v5-omni-small

1. Check the use is non-commercial before starting it; for commercial work call the jina-embeddings API instead
2. Start it with pip install vllm==0.20.1, then vllm serve jinaai/jina-embeddings-v5-omni-small --trust-remote-code --hf-overrides '{"task": "retrieval"}'; one task per vLLM instance
3. Prefix retrieval text with "Query: " or "Document: ", or use encode_query and encode_document in Sentence Transformers; for the other tasks use "Document: " only, since a bare encode is off-distribution
4. Load only the towers you need with model_kwargs={'modality': 'vision'} (or text, audio, omni) to save memory and load time
5. Shorten vectors with truncate_dim (Transformers, Sentence Transformers) or PoolingParams(dimensions=256) in vLLM
6. Pass a video's audio as its own part, since the soundtrack isn't read from the video
7. Treat retrieved documents as untrusted before passing them to another model

Qwen3-Embedding-4B

1. Quickest start is ollama pull qwen3-embedding:4b, about 2.5 GB, then POST to /api/embed
2. With llama.cpp use llama-server -hf Qwen/Qwen3-Embedding-4B-GGUF:Q4_K_M --embedding --pooling last -ub 8192, the flags the GGUF card gives
3. Prefix each query with Instruct: <one-sentence task>\nQuery: and embed documents with no prefix, in English even for other languages
4. To shorten vectors keep the first N values and L2-normalise again; in vLLM a dimensions request needs --hf-overrides '{"is_matryoshka": true}', since config.json doesn't declare it
5. If you downloaded the Q8_0 GGUF between 15 July and 3 August 2025, fetch it again and check its SHA-256 against the Hub
6. Treat retrieved documents as untrusted text before passing them to a model