Ornith-1.5-397B vs Qwen3-4B-Instruct-2507

Ornith-1.5-397B

Ornith's flagship coding and terminal-agent cousin, with XML tool calls that vLLM and SGLang parse and 262,144 tokens of context. It needs about 290 GB even at 4 bits, so a multi-GPU server, not the 128 GB reference machine. The main risk is an absent publisher, with no licence file, no safety results and files replaced without a note.

Qwen3-4B-Instruct-2507

A small, fast cousin for instruction following, short tool-calling jobs and long documents on modest hardware, with no thinking step. It runs in about 3 GB at 4 bits on every common runtime, and two providers host it. The main risk is neglect, with no file changes in over a year, no answers from Alibaba and no safety results.

Ornith-1.5-397B

1. Start it with the card's vLLM command at --tensor-parallel-size 8, or ollama run ornith-1.5:397b on a machine with about 300 GB of memory
2. In vLLM pass --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3, and in SGLang --tool-call-parser qwen3_coder --reasoning-parser qwen3, so tool calls and reasoning come back parsed
3. Thinking is on by default. Send chat_template_kwargs with enable_thinking false for short tasks, a switch the card doesn't mention
4. Pin which template your runtime loads, since chat_template.jinja keeps earlier reasoning and the tokenizer_config.json template drops it
5. When downloading the NVFP4 build, fetch only the files its model.safetensors.index.json names (the 25-shard set, 241.7 GB), not the eight extra shards
6. Pin a revision when you download, since Ornith has replaced files in place
7. Validate JSON output yourself, since structured output isn't documented

Qwen3-4B-Instruct-2507

1. Quickest start is ollama run qwen3:4b-instruct, 2.5 GB. Plain qwen3:4b is the Thinking-2507 edition, not this one
2. On our reference machine use BF16 with vllm serve Qwen/Qwen3-4B-Instruct-2507 --max-model-len 262144 --enable-auto-tool-choice --tool-call-parser hermes
3. Use the hermes parser in vLLM or qwen25 in SGLang, the template emits JSON inside tool_call tags
4. Don't pass enable_thinking, this edition never thinks and the card says the flag isn't needed
5. Sampling per the card is temperature 0.7, top_p 0.8, top_k 20 and min_p 0, with presence_penalty up to 2 against repetition
6. Drop the context to 32,768 tokens if memory runs out, as the card suggests
7. Validate JSON yourself or use the runtime's guided decoding, since structured output isn't documented