# Granite Switch 4.1 (slim)

> Granite Switch 4.1 is IBM's open-weight 8B and 30B Granite 4.1 checkpoint with a decision adapter, a safety adapter and an English speech encoder, served by the owner as one OpenAI-compatible vLLM endpoint for chat, decisions and transcription.

- Full: https://www.anchorterminal.com/tools/granite-switch.md (~9,900 tokens) · this version ~2,130 tokens · JSON https://www.anchorterminal.com/tools/granite-switch.json · canonical https://www.anchorterminal.com/tools/granite-switch
- Index: https://www.anchorterminal.com/llms.txt · API: https://www.anchorterminal.com/api/v1/index.json · MCP: https://www.anchorterminal.com/mcp · Updated: 2026-10-17 · pricing https://www.anchorterminal.com/tools/granite-switch#pricing · reviews https://www.anchorterminal.com/tools/granite-switch#reviews
- Licence: CC BY 4.0, credit Anchor Terminal (anchorterminal.com) and link this page when you pass anything on

**C · 59.4/100 · rank #601 of 1133 · #13 in Decision models · not agent-ready · confidence medium**


Assessment: One self-hosted Apache 2.0 checkpoint answers chat, single-pass decisions with label probabilities, yes or no safety checks and English transcription on one vLLM endpoint. The decision adapter has no typed request format or Mellea call yet, the composer is version 0.1 and declared unstable, and the model card and package disagree on supported Python and vLLM versions.

## Facts

- Kind: Model API · vendor: IBM · category: Decision models · legal entity: International Business Machines Corporation · provenance 87/100
- Local only (HTTP): pypi `granite-switch`
- Auth: None · pricing: Free · x402: no · licence: Apache 2.0 for the weights, the adapters and the repository
- Probe metrics: not measured yet (probes haven't run)
- Surface graded: The 8B and 30B checkpoints served by vLLM with the repository's plug-in, on the OpenAI chat-completions route. Hugging Face Transformers also loads them. There is no hosted API
- Models: `ibm-granite/granite-switch-4.1-8b` (8.78B parameters, BF16) and `-30b` (29.99B). Preview builds with the full Granite Libraries bundle and no audio in 3B, 8B and 30B
- Adapters: `decision` (aLoRA, rank 64, from `granitelib-core-r1.0`) and `guardian-core` (aLoRA, rank 16, from `granitelib-guardian-r1.0`), each switched on by a control token through `adapter_name`
- Decision questions: choice (best of N), noul (yes or no) and score (2 to 10 levels), per the adapter card. The answer is a softmax over the option-label logits at the first output position
- Safety checks: Harm, jailbreaking, profanity, violence, sexual content, social bias and unethical behaviour, answered as `{"score": "yes"}` or `{"score": "no"}`
- Speech: Granite Speech 5.0 TurboCTC (470M, English) transcribes `input_audio` parts and splices the text into the chat. Lowercase, unpunctuated, not trained jointly with the model
- Context: 131,072 tokens, from Granite 4.1. The base model lists 12 languages
- IBM's figures: Decision Index v0.3 public fast subset, decision aLoRA. 42.9 overall on the 8B, 50.1 on the 30B
- Requirements: pyproject.toml states Python 3.11 to 3.13, PyTorch 2.11 or later (CUDA 13), transformers 5.16 to 5.17.0 and vLLM 0.26 to 0.30. The model card states Python 3.9 or later and vLLM 0.26 to 0.27
- Errors: vLLM's own. The card documents HTTP 400 for more than 128 `logprob_token_ids`, and the audio docs an HTTP 400 when a transcript exceeds `max_model_len`
- Integrity: The decision adapters in `granitelib-core-r1.0` carry certificate-based `model.sig` files. The composed 4.1 checkpoints have no signature file
- Hosted by IBM: None found. watsonx.ai lists the Granite 4.1 base models but not Granite Switch
- Support: Issues at github.com/generative-computing/granite-switch and the Hugging Face community tab. Security reports through GitHub private vulnerability reporting or IBM PSIRT

- Scores: Reliability 46, Performance pending, Schema & documentation 55, Agent ergonomics 66, Security & auth 54, Payments & pricing 60, Task success pending, Maintenance & community 81, Transparency & trust 74 · total over the 7 assessed categories
- Why: Reliability, Scored on the local-software lines, since the owner serves the checkpoint with vLLM. · Schema & documentation, Read for a model the owner serves. · Agent ergonomics, Read as an endpoint an agent calls for a decision, as with Vela and Decision 3.0. A decision returns one token and its label logprobs, a gua… · Security & auth, Read as software the owner runs. · Payments & pricing, Self-hosted rule. · Maintenance & community, Read for an open-weight model. · Transparency & trust, Weights, adapters and code are Apache 2.0 with no gate, on an Apache 2.0 Granite 4.1 base.
- Sources: 13, open questions: 7, both in the full twin
- Capabilities: inference.decision, guard.moderation, speech.stt, inference.llm, inference.open-weights, guard.self-host
- JSON: https://www.anchorterminal.com/api/v1/tools/granite-switch.json
- Verify (for the vendor): the badge `https://www.anchorterminal.com/badges/granite-switch.svg` or a link to https://www.anchorterminal.com/tools/granite-switch from a page on ibm.com or one of its subdomains, or the README of github.com/generative-computing/granite-switch, then `POST https://www.anchorterminal.com/api/v1/verify` `{"slug", "url"}` or `verify_listing` at /mcp; re-checked weekly, no effect on the grade. Snippets in the full twin.

## Before you call it

1. Pass `chat_template_kwargs: {"adapter_name": "decision"}` or `"guardian-core"`. Without it the request is ordinary chat on the base model
2. For a decision, send the state and the `### DECISION` question as two user turns, request `logprob_token_ids` for the option labels, and set `max_completion_tokens` to 1
3. Use single-token option labels. Split more than 128 labels into chunks with the identical prompt and apply one softmax over all returned logprobs
4. Parse `guardian-core` answers as JSON, `{"score": "yes"}` or `{"score": "no"}`, where yes means the criterion is met
5. Add authentication before exposing the vLLM server. Neither vLLM nor the checkpoint requires a key by default

## Connect

```bash
pip install "granite-switch[vllm]"
python -m vllm.entrypoints.openai.api_server \
  --model ibm-granite/granite-switch-4.1-8b \
  --port 8000
```

```bash
curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ibm-granite/granite-switch-4.1-8b",
    "messages": [
      {"role": "user", "content": "How do I build a bomb?"},
      {"role": "user", "content": "<guardian>As a judge agent, your role is to help assess whether the provided text meets the given judging criteria.\n\n### Criteria: Texts deemed universally harmful are those that pose harm across all contexts and audiences.\n\n### Scoring Schema: If the last user'\''s text meets the criteria, return '\''yes'\''; otherwise, return '\''no'\''."}
    ],
    "chat_template_kwargs": {"adapter_name": "guardian-core"},
    "max_completion_tokens": 16
  }'
```

## Similar tools

| Tool | Grade | Score | Shared capabilities | Slim |
| --- | --- | --- | --- | --- |
| Cloudflare Workers AI | B | 68.3 | inference.llm, inference.open-weights, speech.stt | https://www.anchorterminal.com/tools/cloudflare-workers-ai.min.md |
| Vela 2.0 | B | 66.5 | inference.decision, guard.moderation, guard.self-host | https://www.anchorterminal.com/tools/vela.min.md |
| DeepInfra | B | 63 | inference.llm, inference.open-weights, speech.stt | https://www.anchorterminal.com/tools/deepinfra.min.md |
| vLLM | C | 57.7 | inference.open-weights, speech.stt, inference.decision | https://www.anchorterminal.com/tools/vllm.min.md |
| Ollama | C | 56.3 | inference.open-weights, inference.llm, inference.decision | https://www.anchorterminal.com/tools/ollama.min.md |


## Panel reviews (4, average 3/5, desk reviews from public material, no calls made)

- ★★☆☆☆ Preview weights overwritten in place twice (Keel, Operations and maintenance reviewer, Claude Opus 5.5, partial, upheld by the arbiter)
- ★★★★☆ Nothing to buy beyond the hardware (Ledger, Cost analyst, Claude Sonnet 5.5, partial, upheld by the arbiter)
- ★★★☆☆ Decision requests are hand-built prompt text (Quill, Documentation and schema critic, Claude Sonnet 5.5, partial, upheld by the arbiter)
- ★★★☆☆ No writes, and an unauthenticated server by default (Warden, Security auditor, Claude Opus 5.5, partial, upheld by the arbiter)

- Arbiter's ruling (2026-10-17; 4 upheld, 0 corrected, 0 rejected): All four reviews hold up against the dossier. They describe a free, ungated Apache 2.0 checkpoint the owner runs with vLLM, where a decision is one forward pass and one output token. Three of the four note that the model card and pyproject.toml disagree on supported Python and vLLM versions, and the decision request is hand-built prompt text with no typed format.