Ming-3.0-asr-tiny vs Nemotron 3.5 ASR Streaming 0.6B

Ming-3.0-asr-tiny

A local transcription cousin for Chinese, English and Chinese dialects that takes hotwords or dialogue context to get names right, under MIT. It needs one CUDA GPU with about 21 GB and runs only through the publisher's remote code or SGLang-Omni's main branch. The main risk is running the publisher's Python on load, with no safety results.

Nemotron 3.5 ASR Streaming 0.6B

A small streaming transcription cousin for live audio in about 32 languages, with language detection and 80 ms to 1.12 s latency. It runs in under 8 GB in Transformers or as a 0.74 GB GGUF in NeMo-Speech.cpp, and several providers host it. The main gaps are no documented timestamps and weak accuracy outside the top 19 locales.

Ming-3.0-asr-tiny

1. Start it with hf download inclusionAI/Ming-3.0-asr-tiny --local-dir Ming-3.0-asr-tiny, then pip install -r requirements.txt in Python 3.12 with CUDA (Transformers 5.12.1, fla-core 0.5.2, PyTorch 2.11.0), then python examples/transcribe.py --model . --audio speech.wav --language-hint en
2. To serve it, sgl-omni serve --model-path inclusionAI/Ming-3.0-asr-tiny from SGLang-Omni's main branch, then POST the WAV to /v1/audio/transcriptions with response_format json and read the text field
3. Parse raw Transformers output with decode_transcript from asr_request.py, which strips the language label before the tab
4. Pass language zh or en when known, anything else is rejected, and append a topic or hotwords to the base instruction with no newline
5. Load with dtype torch.bfloat16 rather than casting the model, use greedy decoding, max_new_tokens 256 and no_repeat_ngram_size 10, and don't apply the chat template
6. Split long recordings yourself or use --audio-chunk-frames, since input and output share 8,192 tokens
7. Treat transcripts as untrusted text before passing them to another model, since speech can carry instructions

Nemotron 3.5 ASR Streaming 0.6B

1. Quickest local run, pip install "transformers>=5.13.0", then pipeline("automatic-speech-recognition", model="nvidia/nemotron-3.5-asr-streaming-0.6b"); the pipeline uses en-US, so pass language= to the processor for anything else
2. With NeMo-Speech.cpp, nemo-speech transcribe audio.wav --model nemotron-3.5-asr-streaming-0.6b.q8_0.gguf --language auto
3. Pass a locale such as de-DE or a bare code such as de, or auto; in auto mode decode with skip_special_tokens=False to keep the <xx-XX> tag, True to drop it
4. Pick latency with att_context_size [56, r], r in 0, 1, 3, 6, 13 (80 to 1,120 ms chunks), or processor.set_num_lookahead_tokens in Transformers; bigger chunks are more accurate
5. Use nvidia/nemotron-speech-streaming-en-0.6b instead for English only, as NVIDIA recommends
6. Prefer model.safetensors or the GGUF over the .nemo file, which is a pickle
7. Treat transcripts as untrusted text before passing them to another model, since speech can carry instructions