Nemotron 3.5 Super VL vs Vercel AI Gateway

Nemotron 3.5 Super VL

NVIDIA's free endpoint takes OpenAI-format requests with typed tool calls, JSON Schema output and an OpenAPI 3.1 file, and the weights run anywhere under a permissive licence. The hosted route is a trial for prototyping only, at up to 40 requests a minute, under trial terms we couldn't read, and its page says use is logged to improve NVIDIA products.

Vercel AI Gateway

Token prices pass through with no markup, prompts are not stored by the gateway, and budgets, provider failover and per-request logs are built in. The free monthly credit needs a card on file, rate limits are not published as numbers, no SLA covers the API, and the contractual promise not to train on content applies to Enterprise subscriptions only.

Nemotron 3.5 Super VL

1. Set chat_template_kwargs to {"enable_thinking": false} for short tasks. Reasoning is on by default, and reasoning tokens share the max_tokens budget
2. Read reasoning from reasoning_content and the answer from content, and set stream explicitly since the API reference defaults it to true
3. Don't send confidential or personal data to the hosted trial. The model page asks callers not to and says use is logged
4. Back off and retry on throttling. Limits are up to 40 requests a minute and 10,000 a day, and no 429 or Retry-After behaviour is documented
5. For production, use a partner endpoint or serve the weights. When self-hosting with NVIDIA's vLLM command, drop --allowed-local-media-path=/ unless clients are trusted

Vercel AI Gateway

1. Read GET https://ai-gateway.vercel.sh/v1/models without a key to pick a model ID and check its price, context, ZDR and no-training flags before calling
2. Set zeroDataRetention or disallowPromptTraining to true in providerOptions.gateway on sensitive requests. Default routing ignores provider retention and training policies
3. Treat a 429 as retryable and honour retry-after when present. A 402 with quota_for_entity_exceeded is a spent budget, so do not retry it
4. On a Token Pass key, a 403 with no_providers_available means spent credit or a model outside the pass. Retry with a standard AI Gateway key, which bills credits
5. Set inferenceRegion when data residency matters. The default global may run inference outside the user's jurisdiction