Hy3 vs Muse Glimmer 30B

Hy3

Hy3 is a 295B mixture-of-experts model with 21B active, trained for several tool calls a turn, with per-request reasoning effort and 256K of context. It needs an 8-GPU server for FP8 or about 200 GB for a 4-bit build. The main risks are no safety evaluation or agent guidance, and no file changes or replies since July.

Muse Glimmer 30B

A dense 30B multimodal cousin for local agent and coding work with images, documented tool calls and adjustable reasoning. Meta's 17 GB GGUF fits a 24 GB card and runs in llama.cpp, Ollama, vLLM, SGLang and Transformers. The main risks are one tool call per turn, no documented JSON output and a usage policy that adds limits to Apache 2.0.

Hy3

1. Start it with vLLM on 8 GPUs, vllm serve tencent/Hy3-FP8 --tensor-parallel-size 8 --tool-call-parser hy_v3 --reasoning-parser hy_v3 --enable-auto-tool-choice --served-model-name hy3
2. In SGLang use --tool-call-parser hunyuan --reasoning-parser hunyuan, as the card shows
3. Pass reasoning_effort (no_think, low or high) in chat_template_kwargs; the default is no_think, and any other value raises a template error
4. With the vLLM reasoning parser the chain of thought comes back in message.reasoning and the answer in message.content
5. Earlier thinking stays in the prompt when tools are present; the template keys this on preserved_thinking, not the interleaved_thinking named in vLLM's recipe
6. Use temperature 0.9 and top_p 1.0, the card's recommended sampling
7. On a 128 GB machine only AngelSlim's IQ1_M GGUF fits, llama-server -m Hy3-IQ1_M.gguf -ctk q8_0 -ctv q8_0 -fa on -c 65536, with llama.cpp b11768 or later; at 89.4 GB × 1.2 it sits right at the 107.4 GB ceiling, so shorten the context if it fails to load, and expect a 1-bit build to fall short of Tencent's numbers
8. Without the hardware, call it through Featherless AI or DeepInfra via Hugging Face's inference providers

Muse Glimmer 30B

1. Start it with llama serve -hf meta-models/Muse-Glimmer-30B-GGUF --hf-file Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf -ngl 99 -c 131072 --host 127.0.0.1 --port 8080 --chat-template-kwargs '{"reasoning_strength":"high"}' on llama.cpp b10353 or later, or ollama run muse-glimmer on Ollama 0.32.7 or later
2. In vLLM pass --tool-call-parser muse_glimmer --reasoning-parser muse_glimmer, in SGLang muse for both, in ExecuTorch --tool-parser atem. Without them tool calls come back as plain text
3. Set reasoning per request with chat_template_kwargs.reasoning_strength (low, medium, high or xhigh). llama.cpp and ExecuTorch don't read reasoning_effort, and ExecuTorch rejects it with a 400
4. Expect one tool call per turn and return each result before the next. ExecuTorch's server rejects tool_choice="required"
5. Keep temperature 1.0, top_p 0.95 and top_k 64, don't run greedy, and never stop on <|eom|>
6. Leave max_tokens room for reasoning, since in llama.cpp a request cut off mid-thought returns empty content with finish_reason length
7. For images add Meta's 1.4 GB mmproj GGUF, which takes the build to about 22 GB with the runtime's fifth