{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/open-weights.speech-input/",
    "json": "https://www.anchorterminal.com/niches/open-weights.speech-input/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/open-weights.speech-input/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/open-weights.speech-input/index.min.md"
  },
  "markdown": "One company's docs show a full fit.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 1 | 4 | 90 | 6 | 3 |\n\n## What counts as done\n\nThe model takes the audio clip as input and returns a transcript.\n\nMeasured, for our own tests: word error rate within 3 points of Whisper Medium on a 50-clip sample, real-time factor under 0.1, and no HTTP 500 from stray control tokens.\n\nHow agents ask for it: a local Whisper alternative; dictation in my desktop app without the cloud; summarise this audio file locally; voice in, text out on a laptop\n\n## Does the job\n\nThe docs show it meets the test above, and nothing we read says otherwise.\n\n- **Gemma 4 E4B IT (Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author))**: acc-44 measured E4B Q4_K_M at 13.82 percent WER on 50 LibriSpeech test-other clips through llama-server (Whisper LargeV3Turbo 11.48); acc-45 transcribed three languages and calls E4B a genuine Whisper alternative. Audio works in llama.cpp, not Ollama.\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **Apertus v1.5 8B (ETH Zurich and EPFL (the Swiss National AI Institute, named in the Acceptable Use Policy))**: Ships a WavTokenizer for audio input, but no llama.cpp or Ollama path exists and only the publisher's vLLM fork serves it; no WER figure found.\n- **Gemma 4 12B IT (Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author))**: Google confirms audio on E2B, E4B and 12B (discussion 132) and the llama.cpp guide says audio works on the 12B; Ollama passes text and images only, and no WER account covers the 12B.\n- **Gemma 4 E2B IT (Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author))**: Audio is native on E2B and acc-44 measured 13.15 percent WER at BF16 on LibriSpeech test-other, but the Q8_0 build emitted stray channel tokens that gave HTTP 500s and acc-45 found E2B garbled three-language audio.\n- **MiniCPM-o 4.5 (OpenBMB (named as the MiniCPM-o team on the technical report; the GitHub LICENSE is the unfilled Apache template))**: An audio GGUF is shipped, but the card routes speech through OpenBMB's llama.cpp-omni fork or Transformers with trust_remote_code; mainline llama.cpp and Ollama are listed for text and images.\n- **Nemotron 3 Nano Omni 30B A3B Reasoning (NVIDIA Corporation)**: Takes wav and mp3 up to an hour, HF-ASR WER 5.95 (NVIDIA's run), but audio is documented only on vLLM with vllm[audio] and a 32 GB-plus GPU; Ollama lists text and image input only, and it is English only.\n- **Qwen3-Omni-30B-A3B-Instruct (Alibaba Cloud (carried from the 20 October 2026 pilot; this model's files name no copyright holder))**: Audio in through vLLM's Thinker and llama.cpp's Qwen3 Omni GGUF with audio input, but the card gives no WER against Whisper and the build is 20 GB, not the small model the use case states.\n\n- not this job: Agents-A1-4B (Shanghai Artificial Intelligence Laboratory (the InternAgent team there, per the card's recommended system prompt and the template's default prompt); the GitHub LICENSE leaves the copyright line as the Apache template): config.json has a vision_config but no audio_config; the model takes no audio.\n- not this job: Apodex-1.1-mini (Apodex (the site's copyright line reads 2026 Apodex)): No audio_config in config.json; vision only.\n- not this job: BigBang-v1 (The Endless Frontier (endless-frontier)): No audio_config in config.json.\n- not this job: Carbon-X (Hugging Face (the HuggingFaceBio organisation, named Hugging Face Biology Research on the Hub)): Text only; no audio_config.\n- not this job: Cosmos-Reason2-2B (NVIDIA Corporation): Image and video only; no audio path in the config.\n- not this job: Darwin-36B-Opus (VIDRAFT Inc. (named on vidraft.net); the card's citation names FINAL-Bench and VIDRAFT_LAB): Text only.\n- not this job: DeepSeek-V3.2 (DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)): Text only.\n- not this job: DeepSeek V4.1 Flash (DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)): No audio_config; images and text only.\n- not this job: DeepSeek-V4-Flash-0731 (DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)): Text only.\n- not this job: DiffusionGemma 26B A4B IT (Google LLC (RDAP registrant of gemma.ai; the card names Google DeepMind as author)): No audio_config.\n- not this job: Gemma 4 26B A4B IT (Google LLC (RDAP registrant of gemma.ai; the card names Google DeepMind as author)): The 26B A4B has no audio encoder; Google's guide and discussion 132 limit audio to E2B, E4B and 12B.\n- not this job: Gemma 4 31B IT (Google LLC (the registrant of gemma.ai; the card names Google DeepMind as author)): The 31B has no audio tower; Google confirmed in discussion 132 that MP3 input returns HTTP 400.\n- not this job: GLM-4.5-Air (JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai, per the first batch's check on 21 October 2026)): Text only.\n- not this job: GLM-4.7-Flash (JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai)): Text only.\n- not this job: GLM-5.3 (JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai). The licence names Z.AI): Text only.\n- not this job: GLM-5.3-Flash (Z.AI Co., Ltd (named in the licence's copyright line)): No audio_config.\n- not this job: gpt-oss-120b (OpenAI (named as publisher on the card; the openai-api listing's terms name OpenAI OpCo, LLC)): Text only.\n- not this job: gpt-oss-20b (OpenAI (named in USAGE_POLICY and the model card; the LICENSE file's copyright line is the unfilled Apache template)): Text only.\n- not this job: Granite 4.2 3B (International Business Machines Corporation (per IBM's naming guidance, as read for granite-guardian)): Text only.\n- not this job: Hy3 (Tencent (named in the LICENSE file's copyright line)): Text only.\n- not this job: Inkling (Thinking Machines Lab, Inc. (named on its model card page and in its training-content summary)): Takes 16 kHz audio, but nothing under 270 GB runs it and no llama.cpp release lists the architecture.\n- not this job: Inkling-Small (Thinking Machines Lab, Inc. (named on its model card page and in its training-content summary)): Takes 16 kHz WAV, but nothing under 75 GB runs it and llama.cpp's release list lacks the architecture.\n- not this job: InternVL3.5-GPT-OSS-20B-A4B-Preview (Shanghai AI Laboratory (OpenGVLab), not named in the repository): No audio_config.\n- not this job: Kimi K3 (Moonshot AI (the licence names Moonshot AI; the Kimi Platform terms name Moonshot AI PTE. LTD.)): No audio_config.\n- not this job: Kolibri-1 (Aleph Alpha GmbH): Text only.\n- not this job: Kronos (No company. Yu Shi publishes it from the GitHub account shiyu-coder and the Hugging Face user account NeoQuasar (named ShiYu), with co-authors at Tsinghua University per the paper): No audio input.\n- not this job: Laguna M.1 (Poolside, Inc. (named in Poolside's Terms of Use; the LICENSE.md copyright line reads Poolside)): No audio input.\n- not this job: Laguna S 2.1 (Poolside, Inc. (named in Poolside's Terms of Use)): No audio input.\n- not this job: Laguna XS 2.1 (Poolside, Inc. (named in Poolside's Terms of Use)): No audio input.\n- not this job: LBS-27B (inclusionAI, Ant Group (named in the site footer of inclusion-ai.org); the work is credited to authors from Wuhan University of Technology, CASIA, Zhejiang University and Ant Group): No audio input.\n- not this job: LFM2.5-2.6B (Liquid AI, Inc. (the licensor named in the LFM Open License)): No audio input.\n- not this job: Ling-3.0-tiny (Ant Intelligence (Hang Zhou) Technology Co., Ltd (named as provider in the training-content summary); inclusionAI, Ant Group in the site footer of inclusion-ai.org): No audio input.\n- not this job: MiMo-V2.6 Distill Qwen 9B (Xiaomi (the card and report name Xiaomi MiMo; no LICENSE file names a copyright holder)): config.json has no audio encoder.\n- not this job: MiniCPM5-2B (OpenBMB (named in the GitHub LICENSE's copyright line)): No audio input.\n- not this job: MiniMax-M2.5 (MiniMax AI (named in the licences; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)): No audio input.\n- not this job: MiniMax-M2.7 (MiniMax (named in the licence's copyright line; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)): Text only; no audio tower in config.json.\n- not this job: MiniMax-M3.1-Flash (MiniMax (named in the licence's copyright line; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)): Card lists text, image and video inputs only.\n- not this job: Mistral Large 4 (Mistral AI (RCS Paris 952 418 325)): Text and image input only.\n- not this job: Muse Glimmer 30B (Meta (the verified Hugging Face organisation is \"Meta Inc.\", the card names Meta Superintelligence Lab, and the licence carries no copyright line)): The card says audio input and output are not supported.\n- not this job: Nanbeige4.3-33B (BOSS Zhipin (Kanzhun Limited), unconfirmed): Text only; no audio tower.\n- not this job: NVIDIA Nemotron 3.5 Bolt E4B 4GB (NVIDIA Corporation): NVIDIA documents text input only; the BF16 config keeps Gemma 4's audio_config but no audio projector or result is published.\n- not this job: Nemotron 3.5 Lightning 30B A3B (NVIDIA Corporation): Text only.\n- not this job: Nemotron 3 Nano 30B A3B (NVIDIA Corporation): Text only.\n- not this job: Nemotron 3 Nano 4B (NVIDIA Corporation): Text only.\n- not this job: Nemotron 3 Super 120B A12B (NVIDIA Corporation): Text only.\n- not this job: Nemotron 3 Ultra 550B A55B (NVIDIA Corporation): Text only.\n- not this job: occamy-1.0 (Accio (Accio-Lab)): Text, image and video only; no audio tower in config.json.\n- not this job: Olmo 3 7B Think (The Allen Institute for Artificial Intelligence (Ai2), a 501(c)(3) nonprofit, per its Responsible Use Guidelines): Text only.\n- not this job: Ornith-1.5-35B-A3B (Ornith): No audio input.\n- not this job: Ornith-1.5-397B (Ornith): No audio input.\n- not this job: Ornith-1.5-9B (Ornith): No audio input.\n- not this job: Qwen3-30B-A3B-Instruct-2507 (Alibaba Cloud (named in the licence's copyright line)): Text only.\n- not this job: Qwen3-4B-Instruct-2507 (Alibaba Cloud (named in the licence's copyright line)): Text only; no audio tower in config.json.\n- not this job: Qwen3.5-0.8B (Alibaba Cloud (named in the licence's copyright line)): No audio tower in config.json.\n- not this job: Qwen3.5-122B-A10B (Alibaba Cloud (named in the licence's copyright line)): No audio tower in config.json.\n- not this job: Qwen3.5-2B (Alibaba Cloud (named in the licence's copyright line)): No audio tower in config.json.\n- not this job: Qwen3.5-397B-A17B (Alibaba Cloud (named in the licence's copyright line)): No audio tower in config.json.\n- not this job: Qwen3.5-4B (Alibaba Cloud (named in the licence's copyright line)): No audio tower in config.json.\n- not this job: Qwen3.5-9B (Alibaba Cloud (named in the licence's copyright line)): No audio tower in config.json.\n- not this job: Qwen3.6-27B (Alibaba Cloud (named in the licence's copyright line)): No audio tower in config.json.\n- not this job: Qwen3.6-35B-A3B (Alibaba Cloud (named in the licence's copyright line)): No audio tower in config.json.\n- not this job: Qwen3.8-27B (Alibaba Cloud (named in the licence's copyright line)): No audio tower in config.json.\n- not this job: Qwen3.8-Flash-Next (Alibaba Cloud (Singapore) Private Limited, the registrant of qwen.ai. The licence names Qwen as copyright holder): No audio tower in config.json.\n- not this job: Qwen3-Coder-30B-A3B-Instruct (Alibaba Cloud (named in the licence's copyright line)): Text only; no audio tower in config.json.\n- not this job: Qwen3-Coder-Next (Alibaba Cloud (the Qwen team's employer; no LICENSE file names a copyright holder for this model)): Text only; no audio tower in config.json.\n- not this job: Qwen3-VL-2B-Instruct (Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder)): No audio tower in config.json.\n- not this job: Qwen3-VL-30B-A3B-Instruct (Alibaba Cloud (per the pilot's RDAP check of qwen.ai; this model's files name no copyright holder)): No audio tower in config.json.\n- not this job: Qwen3-VL-32B-Instruct (Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder)): No audio tower in config.json.\n- not this job: Qwen3-VL-4B-Instruct (Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder)): No audio tower in config.json.\n- not this job: Qwen3-VL-8B-Instruct (Alibaba Cloud (per the pilot's RDAP check of qwen.ai; this model's files name no copyright holder)): No audio tower in config.json.\n- not this job: Qwopus3.8-27B-Flash-V2 (Jackrong, an individual Hugging Face account): No audio tower in config.json.\n- not this job: Recursive Flint (Recursive Superintelligence, Inc. (the Hugging Face organisation's full name)): Text only; no audio tower in config.json.\n- not this job: Ripple-65B Preview (Ricursive Intelligence, Inc.): Text only; no audio tower in config.json.\n- not this job: SmolLM3-3B (Hugging Face, Inc.): Text only; no audio tower in config.json.\n- not this job: Spark-X2.6-4B (XHToken (the licence's copyright line); the Hugging Face organisation's name is SparkLLM): Text only; no audio tower in config.json.\n- not this job: Swift 1.5 Qwen3.8-27B (UkisAI): No audio tower in config.json.\n- not this job: Swift 1.5 Qwen3.8-Flash-Next (UkisAI): No audio tower in config.json.\n- not this job: ThinkingCap (Qwen3.8 27B) (BottleCap AI (named in the licence's copyright line)): The base Qwen3.8-27B has no audio tower.\n- not this job: Baidu Unlimited-OCR-2 (Baidu, Inc.): No audio tower in config.json.\n\n28 listings in the category were added after this reading and aren't marked yet.\n\n## What stops agents finishing\n\n- the size the user picked has no audio tower (Gemma 4 31B, 26B)\n- Ollama does not pass audio, only llama.cpp does\n- E2B at Q8 emits stray channel tokens that break the template parser\n\n## Around it\n\nBefore: audio.capture. After: open-weights.tool-calling-assistant, text-to-speech.speak.\n\n## Where this comes from\n\nWeight 6 of 100 in Open-weight models: 3 of 51 accounts, all on Gemma 4 small sizes; no account for Qwen3-Omni or Nemotron Omni\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/open-weights.speech-input.json, or the MCP tool get_use_case with id open-weights.speech-input. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Transcribe or understand speech on device",
        "url": ""
      }
    ],
    "description": "Transcribe or understand speech on device: 1 companies fully do it, 4 with a limit, 90 carry the tag. Every listing in Open-weight models with the reason and a source.",
    "facts": null,
    "h1": "Transcribe or understand speech on device",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/open-weights.speech-input/",
    "published": "",
    "section": "agents",
    "title": "Transcribe or understand speech on device | Anchor Terminal",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/open-weights.speech-input/"
  },
  "tokens": {
    "markdown": 3950,
    "slim": 430
  },
  "version": 1
}
