OpenAI embeddings vs Voyage AI embeddings and rerankers

OpenAI embeddings

text-embedding-3-small at $0.02 per million tokens, $0.01 through the Batch API. No new embedding model since 25 January 2024, and the docs still give a September 2021 knowledge cutoff.

Voyage AI embeddings and rerankers

200 million free tokens per current model, then $0.02 to $0.12 per million. Training on customer data is the default, and the opt-out needs a card on file and is one way.

OpenAI embeddings

1. Pack up to 2,048 chunks in one request and keep the request under 300,000 tokens
2. Count tokens before sending. An input over 8,192 tokens is rejected, not truncated
3. Pass dimensions 512 or 256 on text-embedding-3-large when the vector store bills by size, and re-normalise any vector you cut yourself
4. Split a Batch API index job into batches of under 50,000 inputs. It's half price with a 24-hour window
5. Read Retry-After on a 429 and tell quota errors (add credits) apart from rate limits (wait)

Voyage AI embeddings and rerankers

1. Opt the organisation out of training before sending anything private. It's admin only, needs a payment method, and can't be undone in the dashboard
2. Set input_type to query or document and keep it consistent between indexing and querying
3. Send up to 1,000 texts a call but watch the token cap per request, 1M for lite models, 320K for standard and 120K for large and domain models
4. Ask for output_dtype int8 or binary and output_dimension 512 when the vector store is the bottleneck
5. Use rerank-3-lite over the top 100 from a cheap first pass, at $0.02 per million tokens