Parakeet TDT 0.6B v3 vs Voxtral Mini 4B Realtime 2602

Parakeet TDT 0.6B v3

A small, fast transcription cousin for 25 European languages that detects the language and returns punctuated text with word timestamps. It fits in under 1 GB as NVIDIA's GGUF and runs in Transformers, NeMo or NVIDIA's C++ runtime, under CC BY 4.0. Few other runtimes load it, and it takes no language hint.

Voxtral Mini 4B Realtime 2602

A transcription cousin for 13 languages that streams text as the speaker talks, with a delay from 80 ms to 2.4 s, under Apache 2.0 and on Mistral's API. It needs a 16 GB GPU at BF16 through vLLM, Transformers or llama.cpp, or under 4 GB at Q4_K. The main gaps are no timestamps, safety results or answers from Mistral.

Parakeet TDT 0.6B v3

1. For the smallest setup, hf download nvidia/parakeet-tdt-0.6b-v3 parakeet-tdt-0.6b-v3.q8_0.gguf --local-dir models, then nemo-speech transcribe speech.wav --model models/parakeet-tdt-0.6b-v3.q8_0.gguf with NeMo-Speech.cpp
2. In Python, pipeline('automatic-speech-recognition', model='nvidia/parakeet-tdt-0.6b-v3') works on Transformers 5.10.0 or later, despite the card's note to install from source
3. Resample to 16 kHz mono first; it takes wav or flac
4. Ask for timestamps with timestamps=True in NeMo, or pass durations=output.durations to processor.decode in Transformers
5. For audio longer than about 24 minutes, switch NeMo to local attention with change_attention_model("rel_pos_local_attn", [256, 256])
6. Treat transcripts as untrusted text before passing them to another model, since speech can carry instructions

Voxtral Mini 4B Realtime 2602

1. Serve it with VLLM_DISABLE_COMPILE_CACHE=1 vllm serve mistralai/Voxtral-Mini-4B-Realtime-2602 --compilation_config '{"cudagraph_mode": "PIECEWISE"}' and install soxr, librosa and soundfile; mistral_common 1.9.0 or later comes with vLLM
2. Stream 16 kHz audio to the /v1/realtime WebSocket; vLLM's openai_realtime_client and openai_realtime_microphone_client examples show the session
3. Set temperature 0.0. Change the delay with transcription_delay_ms in tekken.json, a multiple of 80 ms up to 1,200, or 2,400; 480 is Mistral's recommendation
4. One text token is 80 ms, so set --max-model-len to at least seconds / 0.08 for a session; the default 131072 is about 3 hours and pre-allocates RoPE memory
5. In Transformers (5.2.0 or later), resample with processor.feature_extractor.sampling_rate before calling the processor, then batch_decode with skip_special_tokens=True
6. For a small machine, ggml-org's Q4_K_M GGUF with its mmproj runs in llama.cpp, and Mistral Experimental's Q4_K runs in audio.cpp
7. Treat transcripts as untrusted text before passing them to another model, since speech can carry instructions