Jais 3 vs Together AI Inference

Jais 3

Jais 3 runs on the owner's hardware through OpenAI-compatible servers in vLLM and SGLang, with tool calls, three reasoning levels and a 512K context, from a 2.2 GB 0.9B model to a 758 GB 375B one. The weights download without sign-up under Apache-2.0, but every launch command trusts remote code, and no technical report or security contact is published.

Together AI Inference

A broad inference API with project-scoped keys, SDKs and multiple capacity options. Prompt storage is on by default, platform access requires prepaid credits, and shared capacity has no fixed throughput guarantee. Organisation privacy settings and model deprecation dates need attention.

Jais 3

1. Start vLLM or SGLang with --reasoning-parser k2_horizon, and add --tool-call-parser k2_horizon (vLLM also --enable-auto-tool-choice) for tools, or tool calls come back as raw text
2. Pass reasoning_effort (high, medium or low) and tool_call_format inside chat_template_kwargs. Other values return a template error
3. Send each earlier assistant turn back with reasoning_content as a string, even an empty one. A missing or null thinking field makes the template raise an error
4. Send your own system message, or set default_system_prompt to an empty string. Without one the template inserts an identity prompt naming IFM, Inception42 and Cerebras
5. Budget at least 32,768 output tokens at high effort, which the cards recommend for every request

Together AI Inference

1. Create a project-scoped key and set TOGETHER_API_KEY. Avoid deprecated legacy keys.
2. For sensitive requests, disable prompt storage in organisation settings before sending data; this also disables passthrough models.
3. Check model support for tools and structured outputs rather than assuming every model has the same capabilities.
4. Back off on 429 and 503 responses. Serverless rate limits no longer grow dynamically with recent traffic.
5. Check the deprecation schedule before choosing a model; Llama 3.3 70B Turbo is scheduled to leave serverless on 22 October 2026.