Qwen3.5-397B-A17B vs Qwen3-VL-30B-A3B-Instruct

Qwen3.5-397B-A17B

Alibaba's biggest open Qwen3.5, a cousin for hard coding, agent and vision work. Even at 4 bits it needs about 270 GB, so most agents will call it through Alibaba's Model Studio or one of the many providers that host it. The main risk is silence, with no safety results, no agent guidance and template reports without a reply.

Qwen3-VL-30B-A3B-Instruct

A mid-sized vision-language cousin for screenshots, documents, charts and video that runs fast for its size, with about 3B active parameters. It needs about 23 GB for Alibaba's Q4_K_M GGUF with its vision file, loads in every major runtime under plain Apache 2.0, and three providers host it. With no published safety work, add guardrails before it drives a GUI.

Qwen3.5-397B-A17B

1. The card's server command is vllm serve Qwen/Qwen3.5-397B-A17B --port 8000 --tensor-parallel-size 8 --max-model-len 262144 --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder, and the FP8 build halves the memory
2. It doesn't fit our 128 GB reference machine at any precision, so call a hosted endpoint instead (Model Studio's Qwen3.5-Plus, or a provider serving Qwen/Qwen3.5-397B-A17B)
3. On Model Studio pass "enable_thinking": false directly, while vLLM and SGLang take "chat_template_kwargs": {"enable_thinking": false}
4. Sampling per the card is temperature 0.6, top_p 0.95 and top_k 20 when thinking, and temperature 0.7, top_p 0.8, top_k 20 and presence_penalty 1.5 without
5. For multi-token prediction in vLLM add --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'
6. Keep the context at 128K or more if you rely on thinking, and turn on YaRN only past 262,144 tokens
7. Validate any JSON it returns, since structured output isn't documented

Qwen3-VL-30B-A3B-Instruct

1. Start it with llama-server -hf Qwen/Qwen3-VL-30B-A3B-Instruct-GGUF:Q4_K_M --jinja -c 32768, or ollama run qwen3-vl:30b-a3b-instruct
2. In Ollama the plain qwen3-vl:30b tag is the Thinking edition, so name 30b-a3b-instruct
3. With the GGUF, load the mmproj vision file too, or images won't be read
4. Parse tool calls as JSON inside <tool_call> tags, or use SGLang's --tool-call-parser qwen
5. Use the shipped sampling, temperature 0.7, top_p 0.8 and top_k 20, from generation_config.json
6. Validate JSON output yourself, since structured output isn't documented
7. vLLM needs 0.11.0 or later, Transformers 4.57.0 or later and llama.cpp b6907 or later