Niches · Open-weight models

Transcribe or understand speech on device

One company's docs show a full fit.

1companies, full fit
4companies, fit with a limit
90companies carry the tag
6share of the category's work
3first-hand accounts

What counts as done

The model takes the audio clip as input and returns a transcript.

Measured, for our own tests: word error rate within 3 points of Whisper Medium on a 50-clip sample, real-time factor under 0.1, and no HTTP 500 from stray control tokens. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: a local Whisper alternative, dictation in my desktop app without the cloud, summarise this audio file locally, voice in, text out on a laptop

Does the job 1

The docs show it meets the test above, and nothing we read says otherwise.

ListingWhy
Gemma 4 E4B IT B
Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author)
acc-44 measured E4B Q4_K_M at 13.82 percent WER on 50 LibriSpeech test-other clips through llama-server (Whisper LargeV3Turbo 11.48); acc-45 transcribed three languages and calls E4B a genuine Whisper alternative. Audio works in llama.cpp, not Ollama. source

Does the job, with a limit 6

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
Apertus v1.5 8B C
ETH Zurich and EPFL (the Swiss National AI Institute, named in the Acceptable Use Policy)
Ships a WavTokenizer for audio input, but no llama.cpp or Ollama path exists and only the publisher's vLLM fork serves it; no WER figure found. source
Gemma 4 12B IT BB
Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author)
Google confirms audio on E2B, E4B and 12B (discussion 132) and the llama.cpp guide says audio works on the 12B; Ollama passes text and images only, and no WER account covers the 12B. source
Gemma 4 E2B IT C
Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author)
Audio is native on E2B and acc-44 measured 13.15 percent WER at BF16 on LibriSpeech test-other, but the Q8_0 build emitted stray channel tokens that gave HTTP 500s and acc-45 found E2B garbled three-language audio. source
MiniCPM-o 4.5 C
OpenBMB (named as the MiniCPM-o team on the technical report; the GitHub LICENSE is the unfilled Apache template)
An audio GGUF is shipped, but the card routes speech through OpenBMB's llama.cpp-omni fork or Transformers with trust_remote_code; mainline llama.cpp and Ollama are listed for text and images. source
Nemotron 3 Nano Omni 30B A3B Reasoning C
NVIDIA Corporation
Takes wav and mp3 up to an hour, HF-ASR WER 5.95 (NVIDIA's run), but audio is documented only on vLLM with vllm[audio] and a 32 GB-plus GPU; Ollama lists text and image input only, and it is English only. source
Qwen3-Omni-30B-A3B-Instruct D
Alibaba Cloud (carried from the 20 October 2026 pilot; this model's files name no copyright holder)
Audio in through vLLM's Thinker and llama.cpp's Qwen3 Omni GGUF with audio input, but the card gives no WER against Whisper and the build is 20 GB, not the small model the use case states. source

Doesn't do this job 79

Listings in the category that do something else, with the reason.

ListingWhy
Agents-A1-4B C
Shanghai Artificial Intelligence Laboratory (the InternAgent team there, per the card's recommended system prompt and the template's default prompt); the GitHub LICENSE leaves the copyright line as the Apache template
config.json has a vision_config but no audio_config; the model takes no audio. source
Apodex-1.1-mini C
Apodex (the site's copyright line reads 2026 Apodex)
No audio_config in config.json; vision only. source
BigBang-v1 C
The Endless Frontier (endless-frontier)
No audio_config in config.json. source
Carbon-X C
Hugging Face (the HuggingFaceBio organisation, named Hugging Face Biology Research on the Hub)
Text only; no audio_config. source
Cosmos-Reason2-2B C
NVIDIA Corporation
Image and video only; no audio path in the config. source
Darwin-36B-Opus C
VIDRAFT Inc. (named on vidraft.net); the card's citation names FINAL-Bench and VIDRAFT_LAB
Text only. source
DeepSeek-V3.2 C
DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)
Text only. source
DeepSeek V4.1 Flash C
DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)
No audio_config; images and text only. source
DeepSeek-V4-Flash-0731 C
DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)
Text only. source
DiffusionGemma 26B A4B IT D
Google LLC (RDAP registrant of gemma.ai; the card names Google DeepMind as author)
No audio_config. source
Gemma 4 26B A4B IT BB
Google LLC (RDAP registrant of gemma.ai; the card names Google DeepMind as author)
The 26B A4B has no audio encoder; Google's guide and discussion 132 limit audio to E2B, E4B and 12B. source
Gemma 4 31B IT BB
Google LLC (the registrant of gemma.ai; the card names Google DeepMind as author)
The 31B has no audio tower; Google confirmed in discussion 132 that MP3 input returns HTTP 400. source
GLM-4.5-Air C
JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai, per the first batch's check on 21 October 2026)
Text only. source
GLM-4.7-Flash B
JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai)
Text only. source
GLM-5.3 B
JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai). The licence names Z.AI
Text only. source
GLM-5.3-Flash B
Z.AI Co., Ltd (named in the licence's copyright line)
No audio_config. source
gpt-oss-120b BB
OpenAI (named as publisher on the card; the openai-api listing's terms name OpenAI OpCo, LLC)
Text only. source
gpt-oss-20b B
OpenAI (named in USAGE_POLICY and the model card; the LICENSE file's copyright line is the unfilled Apache template)
Text only. source
Granite 4.2 3B C
International Business Machines Corporation (per IBM's naming guidance, as read for granite-guardian)
Text only. source
Hy3 B
Tencent (named in the LICENSE file's copyright line)
Text only. source
Inkling B
Thinking Machines Lab, Inc. (named on its model card page and in its training-content summary)
Takes 16 kHz audio, but nothing under 270 GB runs it and no llama.cpp release lists the architecture. source
Inkling-Small B
Thinking Machines Lab, Inc. (named on its model card page and in its training-content summary)
Takes 16 kHz WAV, but nothing under 75 GB runs it and llama.cpp's release list lacks the architecture. source
InternVL3.5-GPT-OSS-20B-A4B-Preview D
Shanghai AI Laboratory (OpenGVLab), not named in the repository
No audio_config. source
Kimi K3 B
Moonshot AI (the licence names Moonshot AI; the Kimi Platform terms name Moonshot AI PTE. LTD.)
No audio_config. source
Kolibri-1 C
Aleph Alpha GmbH
Text only. source
Kronos C
No company. Yu Shi publishes it from the GitHub account shiyu-coder and the Hugging Face user account NeoQuasar (named ShiYu), with co-authors at Tsinghua University per the paper
No audio input.
Laguna M.1 C
Poolside, Inc. (named in Poolside's Terms of Use; the LICENSE.md copyright line reads Poolside)
No audio input.
Laguna S 2.1 C
Poolside, Inc. (named in Poolside's Terms of Use)
No audio input.
Laguna XS 2.1 C
Poolside, Inc. (named in Poolside's Terms of Use)
No audio input.
LBS-27B C
inclusionAI, Ant Group (named in the site footer of inclusion-ai.org); the work is credited to authors from Wuhan University of Technology, CASIA, Zhejiang University and Ant Group
No audio input.
LFM2.5-2.6B D
Liquid AI, Inc. (the licensor named in the LFM Open License)
No audio input.
Ling-3.0-tiny C
Ant Intelligence (Hang Zhou) Technology Co., Ltd (named as provider in the training-content summary); inclusionAI, Ant Group in the site footer of inclusion-ai.org
No audio input.
MiMo-V2.6 Distill Qwen 9B C
Xiaomi (the card and report name Xiaomi MiMo; no LICENSE file names a copyright holder)
config.json has no audio encoder.
MiniCPM5-2B C
OpenBMB (named in the GitHub LICENSE's copyright line)
No audio input.
MiniMax-M2.5 C
MiniMax AI (named in the licences; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)
No audio input.
MiniMax-M2.7 C
MiniMax (named in the licence's copyright line; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)
Text only; no audio tower in config.json. source
MiniMax-M3.1-Flash C
MiniMax (named in the licence's copyright line; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)
Card lists text, image and video inputs only. source
Mistral Large 4 C
Mistral AI (RCS Paris 952 418 325)
Text and image input only. source
Muse Glimmer 30B BB
Meta (the verified Hugging Face organisation is "Meta Inc.", the card names Meta Superintelligence Lab, and the licence carries no copyright line)
The card says audio input and output are not supported. source
Nanbeige4.3-33B C
BOSS Zhipin (Kanzhun Limited), unconfirmed
Text only; no audio tower. source
NVIDIA Nemotron 3.5 Bolt E4B 4GB C
NVIDIA Corporation
NVIDIA documents text input only; the BF16 config keeps Gemma 4's audio_config but no audio projector or result is published. source
Nemotron 3.5 Lightning 30B A3B B
NVIDIA Corporation
Text only. source
Nemotron 3 Nano 30B A3B B
NVIDIA Corporation
Text only. source
Nemotron 3 Nano 4B C
NVIDIA Corporation
Text only. source
Nemotron 3 Super 120B A12B B
NVIDIA Corporation
Text only. source
Nemotron 3 Ultra 550B A55B B
NVIDIA Corporation
Text only. source
occamy-1.0 C
Accio (Accio-Lab)
Text, image and video only; no audio tower in config.json. source
Olmo 3 7B Think D
The Allen Institute for Artificial Intelligence (Ai2), a 501(c)(3) nonprofit, per its Responsible Use Guidelines
Text only. source
Ornith-1.5-35B-A3B C
Ornith
No audio input.
Ornith-1.5-397B C
Ornith
No audio input.
Ornith-1.5-9B C
Ornith
No audio input.
Qwen3-30B-A3B-Instruct-2507 C
Alibaba Cloud (named in the licence's copyright line)
Text only. source
Qwen3-4B-Instruct-2507 C
Alibaba Cloud (named in the licence's copyright line)
Text only; no audio tower in config.json. source
Qwen3.5-0.8B D
Alibaba Cloud (named in the licence's copyright line)
No audio tower in config.json. source
Qwen3.5-122B-A10B B
Alibaba Cloud (named in the licence's copyright line)
No audio tower in config.json. source
Qwen3.5-2B D
Alibaba Cloud (named in the licence's copyright line)
No audio tower in config.json. source
Qwen3.5-397B-A17B B
Alibaba Cloud (named in the licence's copyright line)
No audio tower in config.json. source
Qwen3.5-4B C
Alibaba Cloud (named in the licence's copyright line)
No audio tower in config.json. source
Qwen3.5-9B C
Alibaba Cloud (named in the licence's copyright line)
No audio tower in config.json. source
Qwen3.6-27B B
Alibaba Cloud (named in the licence's copyright line)
No audio tower in config.json. source
Qwen3.6-35B-A3B B
Alibaba Cloud (named in the licence's copyright line)
No audio tower in config.json. source
Qwen3.8-27B B
Alibaba Cloud (named in the licence's copyright line)
No audio tower in config.json. source
Qwen3.8-Flash-Next C
Alibaba Cloud (Singapore) Private Limited, the registrant of qwen.ai. The licence names Qwen as copyright holder
No audio tower in config.json. source
Qwen3-Coder-30B-A3B-Instruct C
Alibaba Cloud (named in the licence's copyright line)
Text only; no audio tower in config.json. source
Qwen3-Coder-Next C
Alibaba Cloud (the Qwen team's employer; no LICENSE file names a copyright holder for this model)
Text only; no audio tower in config.json. source
Qwen3-VL-2B-Instruct C
Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder)
No audio tower in config.json. source
Qwen3-VL-30B-A3B-Instruct C
Alibaba Cloud (per the pilot's RDAP check of qwen.ai; this model's files name no copyright holder)
No audio tower in config.json. source
Qwen3-VL-32B-Instruct C
Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder)
No audio tower in config.json. source
Qwen3-VL-4B-Instruct D
Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder)
No audio tower in config.json. source
Qwen3-VL-8B-Instruct C
Alibaba Cloud (per the pilot's RDAP check of qwen.ai; this model's files name no copyright holder)
No audio tower in config.json. source
Qwopus3.8-27B-Flash-V2 C
Jackrong, an individual Hugging Face account
No audio tower in config.json. source
Recursive Flint C
Recursive Superintelligence, Inc. (the Hugging Face organisation's full name)
Text only; no audio tower in config.json. source
Ripple-65B Preview D
Ricursive Intelligence, Inc.
Text only; no audio tower in config.json. source
SmolLM3-3B C
Hugging Face, Inc.
Text only; no audio tower in config.json. source
Spark-X2.6-4B C
XHToken (the licence's copyright line); the Hugging Face organisation's name is SparkLLM
Text only; no audio tower in config.json. source
Swift 1.5 Qwen3.8-27B C
UkisAI
No audio tower in config.json. source
Swift 1.5 Qwen3.8-Flash-Next C
UkisAI
No audio tower in config.json. source
ThinkingCap (Qwen3.8 27B) C
BottleCap AI (named in the licence's copyright line)
The base Qwen3.8-27B has no audio tower. source
Baidu Unlimited-OCR-2 C
Baidu, Inc.
No audio tower in config.json. source

28 listings in the category were added after this reading and aren't marked yet.

What stops agents finishing

  • the size the user picked has no audio tower (Gemma 4 31B, 26B)
  • Ollama does not pass audio, only llama.cpp does
  • E2B at Q8 emits stray channel tokens that break the template parser

Around it

Before: audio.capture. After: open-weights.tool-calling-assistant, text-to-speech.speak.

Where this comes from

Weight 6 of 100 in Open-weight models: 3 of 51 accounts, all on Gemma 4 small sizes; no account for Qwen3-Omni or Nemotron Omni

The same as data: /api/v1/use-cases/open-weights.speech-input.json, or the MCP tool get_use_case with id open-weights.speech-input. Other jobs in this category: /api/v1/niches?category=open-weights.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.