Cohere Embed and Rerank vs NVIDIA NeMo Retriever Embedding and Reranking NIMs

Cohere Embed and Rerank

Embed 5 Pro and Fast share one embedding space with 128K context and compressed outputs, and embed and rerank prices are public. Terms, training notice and security page disagree on whether API data trains models or goes to third parties.

NVIDIA NeMo Retriever Embedding and Reranking NIMs

Self-hosted containers with OpenAPI 3.1 files, typed request fields, five embedding output types and a dated end-of-life list. The API has no authentication or rate limiting of its own, production use needs an NVIDIA AI Enterprise licence at $4,500 a GPU a year, and the release notes carry no dates.

Cohere Embed and Rerank

1. Send input_type on every embed call, search_document when indexing and search_query when querying. The endpoint rejects a call without it
2. Batch 96 inputs a call, the maximum, stay under 2,000 inputs a minute, and check every batch returns every embedding type you asked for (the Python SDK merge drops types missing from the first response)
3. Budget rerank by searches, $2.00 per 1,000 on Rerank 4 Fast. One query with up to 100 documents is one search, and a document over 500 tokens counts as several
4. Set max_tokens_per_doc on rerank. The default of 4,096 truncates long documents even on the 32K models
5. Index with embed-v5.0-pro and query with embed-v5.0-fast at the same output_dimension. Cohere suggests 1,024-dimension int8 to cut vector storage

NVIDIA NeMo Retriever Embedding and Reranking NIMs

1. Send input_type as query or passage on every embedding call. Asymmetric models return HTTP 400 without it, and the wrong value lowers retrieval accuracy per the docs
2. Do not send dimensions and embedding_type together, and send only 2048 or nothing for dimensions on nvidia/nemotron-3-embed-1b
3. Poll /v1/health/ready before the first call. The Docker health check can report unhealthy while the NIM is ready, per the 2.3 known issues
4. Check the image tag on NGC before pulling. The guide uses nemotron-3-embed-1b:2.3, and NGC's record for that image listed tags up to 2.2.2 on 8 October 2026
5. Put a proxy with authentication and TLS in front of port 8000, and sort /v1/ranking results yourself as the request has no top-n field