Carbon-X vs Qwen3-Coder-30B-A3B-Instruct

Carbon-X

A DNA specialist cousin that another agent calls for variant-effect and sequence questions, with DNA kept on the machine. Hugging Face's Q4_K_M GGUF needs about 12 GB and Q8_0 about 19 GB, on llama.cpp b11742 or later. It isn't clinically validated, it isn't trained for tool calls or reasoning, and the card contradicts its own GGUF repository on llama.cpp support.

Qwen3-Coder-30B-A3B-Instruct

A fast coding cousin, with 3.3B of 30.5B parameters active per token, for agentic coding tools, repository reading and fill-in-the-middle without a thinking step. It needs about 21 to 23 GB at 4 bits, and several providers host it. The main risk is that it runs commands for agents with no safety results or guidance, and Alibaba doesn't answer users.

Carbon-X

1. Quickest start is llama-server -hf HuggingFaceBio/Carbon-X-GGUF:Q8_0 -ngl 99 -c 40960 --jinja --chat-template-kwargs '{"enable_thinking": false}' --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0, on llama.cpp b11742 or later
2. With vLLM or Transformers pass trust_remote_code, and expect the tokenizer to fetch Qwen/Qwen3-4B-Base at a pinned revision on first load
3. Keep enable_thinking false and the card's sampling (temperature 0.6, top_p 0.95, top_k 20), the settings every reported result used
4. Keep DNA in files and pass sequences by name, compute positions and base changes in code and send them as context, the card says the model miscounts long sequences
5. Only A, C, G and T form 6-mers, so filter N and other IUPAC codes first or they become a single oov token
6. Strip the dna tags from answers if you need bare sequence, llama.cpp keeps them in the text
7. Cap max_tokens for next-base questions, long DNA generation is slow and degrades

Qwen3-Coder-30B-A3B-Instruct

1. Quickest start is ollama run qwen3-coder:30b, 19 GB
2. On our reference machine use BF16 with vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct --max-model-len 262144 --enable-auto-tool-choice --tool-call-parser qwen3_coder, 61 GB of weights fits the GPU share
3. Use the qwen3_coder parser in vLLM or SGLang, not hermes, since calls come back as function and parameter blocks rather than JSON
4. Pass tool-call arguments in the history as objects, the template iterates them as a mapping
5. Sampling per the card is temperature 0.7, top_p 0.8, top_k 20 and repetition_penalty 1.05, with up to 65,536 output tokens
6. For code completion use the fill-in-the-middle tokens <|fim_prefix|>, <|fim_suffix|> and <|fim_middle|> from Qwen's README
7. Sandbox the commands and files it touches, since nothing from Alibaba covers prompt injection