# HyperQwen (slim) > HyperQwen is syv.ai's open-source serving setup for Qwen3.8-27B on one 24 GB NVIDIA GPU. A patch series against a pinned vLLM, a requantisation pipeline and Docker Compose profiles run vLLM's OpenAI-compatible API on port 18020. - Full: https://www.anchorterminal.com/tools/hyperqwen.md (~8,200 tokens) · this version ~1,630 tokens · JSON https://www.anchorterminal.com/tools/hyperqwen.json · canonical https://www.anchorterminal.com/tools/hyperqwen - Index: https://www.anchorterminal.com/llms.txt · API: https://www.anchorterminal.com/api/v1/index.json · MCP: https://www.anchorterminal.com/mcp · Updated: 2026-10-30 · pricing https://www.anchorterminal.com/tools/hyperqwen#pricing · reviews https://www.anchorterminal.com/tools/hyperqwen#reviews - Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data) **C · 56.9/100 · rank #738 of 1239 · #11 in Local AI · not agent-ready · confidence medium** Assessment: HyperQwen packages a patched vLLM 0.32.0 and requantised Qwen3.8-27B weights into one container with tool calling preset and a loopback bind when no key is set. It runs one model family, its speed figures come from one RTX 3090, and it has no versioned releases, changelog or security policy. ## Facts - Kind: HTTP API · vendor: syv.ai (syv-ai on GitHub) · category: Local AI · legal entity: syv.ai (Copenhagen, CVR DK44671247) · provenance 49/100 - Local only (HTTP): oci `ghcr.io/syv-ai/hyperqwen` - Auth: None · pricing: Free · x402: no · licence: Apache-2.0 - Probe metrics: not measured yet (probes haven't run) - Surface listed: The HyperQwen repository at main on 30 October 2026 (README, docs, PATCHES.md, Dockerfile, docker-compose.yml, .env.example, the launchers and the CI workflows), read from raw.githubusercontent.com. Nothing was installed or run - Interfaces: vLLM 0.32.0's OpenAI-compatible HTTP server on port 18020 (`/v1`, `/health`, `/metrics`), served model name `qwen3.8-27b`. Docker Compose profiles `single`, `batch`, `solo` and `prepare`, or a bare-metal venv install. No MCP server and no SDK of its own - Setups: A batch (about 1,035 tokens a second aggregate at 64 concurrent), B single-user default (127 a second, 64k context), C reproduction (381 a second while quoting, 56k), D long context (150k, about 95 to 100 a second), E huge context (240k, 67 a second mixed). Project figures on one RTX 3090 at 250 W - Model files: dbirks/Qwen3.8-27B-W4A16-AutoRound (about 19.5 GB) with lm_head, embeddings and the MTP module requantised to int8 in place, plus syvai/qwen3.8-27b-3090-fast-variant and syvai/Qwen3.8-27B-DFlash2-W4A16 (about 1 GB each), all Apache-2.0. Qwen/Qwen3.8-27B is the base model - Hardware: A 24 GB Ampere or newer NVIDIA GPU, a CUDA 13 driver (580 or later), about 40 GB of disk. Volta and older can't run it. Measured on compute capability 8.6, with community runs on sm80, sm89 and sm120 and at TP=2 and TP=4 - Tool calling: `--enable-auto-tool-choice --tool-call-parser qwen3_coder --tool-strict-level parameter` by default (`TOOLS=1`), `--reasoning-parser qwen3`, and a source build of xgrammar for bounded strings in tool schemas - Network: Downloads from Hugging Face on first start and the image pull from ghcr.io. The Docker image turns off vLLM's usage statistics. The venv launchers leave vLLM's default on - CI: Two GitHub workflows, `patch integrity` (about 40 GPU-less checks) and `docker image` (builds, applies every patch, runs build-time tests, pushes `latest` and `sha-<7>`, rebuilds weekly). Both badges read passing on 30 October 2026 - Repository activity: 646 commits, 31 contributors, 0 open and 146 closed issues, 1 open and 276 closed pull requests, per shields.io on 30 October 2026. No tags, releases or CHANGELOG - Scores: Reliability 59, Performance pending, Schema & documentation 56, Agent ergonomics 65, Security & auth 41, Payments & pricing 60, Task success pending, Maintenance & community 68, Transparency & trust 55 · total over the 7 assessed categories - Why: Reliability, Read with the local-software lines, since HyperQwen runs on the owner's GPU with no hosted service. · Schema & documentation, Read with the API lines, for the local server an agent calls, which is vLLM 0.32.0's OpenAI-compatible server. · Agent ergonomics, Read with the API lines. · Security & auth, Read with the tool checklist. · Payments & pricing, Read with the self-hosted rule. · Maintenance & community, No versioned release. · Transparency & trust, The editorial half. - Sources: 24, open questions: 6, both in the full twin - Capabilities: inference.local, inference.open-weights - JSON: https://www.anchorterminal.com/api/v1/tools/hyperqwen.json - Verify (for the vendor): the badge `https://www.anchorterminal.com/badges/hyperqwen.svg` or a link to https://www.anchorterminal.com/tools/hyperqwen from a page on syv.ai or one of its subdomains, or the README of github.com/syv-ai/HyperQwen, then `POST https://www.anchorterminal.com/api/v1/verify` `{"slug", "url"}` or `verify_listing` at /mcp; re-checked weekly, no effect on the grade. Snippets in the full twin. - Support: https://github.com/syv-ai/HyperQwen/issues; more in the full twin ## Before you call it 1. Call http://127.0.0.1:18020/v1 with model `qwen3.8-27b` and the `VLLM_API_KEY` value, or any string if no key was set 2. Poll `/health` before the first request. A first start downloads and prepares the model and compiles kernels, which can take over 30 minutes 3. Declare the context window your setup uses (65,536 by default, 131,072 for `CTX=long`, 245,760 for `CTX=huge`) so the client doesn't truncate 4. Ask the owner to set `VLLM_API_KEY` before setting `BIND=0.0.0.0`, and to front the port with a proxy that allowlists routes 5. Treat the README's tokens-per-second figures as the project's measurements on one RTX 3090 at 250 W ## Connect ```bash git clone https://github.com/syv-ai/HyperQwen && cd HyperQwen cp .env.example .env docker compose --profile single up -d # one or a few people chatting docker compose --profile batch up -d # API backend, many concurrent requests ``` Full config and headless snippets are in the full page. Through letme (picks today, calling later): https://letme.dev/hyperqwen ## Similar tools | Tool | Grade | Score | Shared capabilities | Slim | | --- | --- | --- | --- | --- | | LocalAI | B | 68.5 | inference.local, inference.open-weights | https://www.anchorterminal.com/tools/localai.min.md | | Lemonade | B | 63.9 | inference.local, inference.open-weights | https://www.anchorterminal.com/tools/lemonade.min.md | | KoboldCpp | C | 61 | inference.local, inference.open-weights | https://www.anchorterminal.com/tools/koboldcpp.min.md | | llama.cpp | C | 60.6 | inference.local, inference.open-weights | https://www.anchorterminal.com/tools/llama-cpp.min.md | | Foundry Local | C | 60.5 | inference.local, inference.open-weights | https://www.anchorterminal.com/tools/foundry-local.min.md | ## Panel reviews (0, desk reviews from public material, no calls made)