Agents-A1-4B vs Hy3

Agents-A1-4B

A small agent cousin for search, research and tool-calling loops, with vision and 262,144 tokens of context. It runs in about 4 GB from InternScience's Q4_K_M GGUF, in vLLM, SGLang, Transformers or llama.cpp. The main risks are an undeclared base, no safety results or agent guidance, and a template that adds its own dated system prompt when you send none.

Hy3

Hy3 is a 295B mixture-of-experts model with 21B active, trained for several tool calls a turn, with per-request reasoning effort and 256K of context. It needs an 8-GPU server for FP8 or about 200 GB for a 4-bit build. The main risks are no safety evaluation or agent guidance, and no file changes or replies since July.

Agents-A1-4B

1. Quickest start is llama-server -hf InternScience/Agents-A1-4B-Q4_K_M-GGUF --jinja, about 3.4 GB with the vision projector
2. With vLLM use --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder, as on the card, and add --language-model-only for text-only work
3. Always send your own system message, or the template inserts InternScience's default prompt with the date fixed at 2026-07-14
4. The card recommends temperature 0.85, top_p 0.95, top_k 20, min_p 0 and presence_penalty 1.1, and there is no generation_config.json to set them
5. Thinking is on by default, send chat_template_kwargs with enable_thinking false for short tasks, a switch the card doesn't mention
6. The config declares a multi-token prediction layer that isn't shipped, so don't turn on MTP speculative decoding
7. Validate JSON output yourself, since structured output isn't documented

Hy3

1. Start it with vLLM on 8 GPUs, vllm serve tencent/Hy3-FP8 --tensor-parallel-size 8 --tool-call-parser hy_v3 --reasoning-parser hy_v3 --enable-auto-tool-choice --served-model-name hy3
2. In SGLang use --tool-call-parser hunyuan --reasoning-parser hunyuan, as the card shows
3. Pass reasoning_effort (no_think, low or high) in chat_template_kwargs; the default is no_think, and any other value raises a template error
4. With the vLLM reasoning parser the chain of thought comes back in message.reasoning and the answer in message.content
5. Earlier thinking stays in the prompt when tools are present; the template keys this on preserved_thinking, not the interleaved_thinking named in vLLM's recipe
6. Use temperature 0.9 and top_p 1.0, the card's recommended sampling
7. On a 128 GB machine only AngelSlim's IQ1_M GGUF fits, llama-server -m Hy3-IQ1_M.gguf -ctk q8_0 -ctv q8_0 -fa on -c 65536, with llama.cpp b11768 or later; at 89.4 GB × 1.2 it sits right at the 107.4 GB ceiling, so shorten the context if it fails to load, and expect a 1-bit build to fall short of Tencent's numbers
8. Without the hardware, call it through Featherless AI or DeepInfra via Hugging Face's inference providers